Thursday, September 12, 2013

[T-SQL] Exclude rows

[T-SQL] Exclude rows


Exclude rows

Posted: 12 Sep 2013 01:00 AM PDT

Hey,Say I have a table with the following rows;Case | Hours | Typexxx | 3 | 1xxx | 3 | 2xxx | 4 | 1xxx | 1 | 1and I do a sum of the hours, I get 11. However, a Type of 1 is a billed item and 2 is a credit item. If something has been credited, the billed item row should be excluded, resulting in a total of 5.I'm shtoock!

Add Table to Different FileGroup

Posted: 12 Sep 2013 12:05 AM PDT

Hello EveryoneI would like to create a new filegroup and add, for now, only one table, in the new filegroup for archiving purposes. I have not had but very little experience in doing this task.I have created the new file and filegroup in the database. The name of the FileGroup is MessageArchive. I would like to create a new table named MessageArchive in the New Filegroup named MessageArchive by using[code="sql"]SELECT * INTO MessageArchiveFROM Messages[/code]But I am missing something on how to define the filegroup that I want to use, which is MessageArchive.After all the data has been copied into the new table in the MessageArchive filegroup, how does one query from that table?Thank you in advance for your assistance, comments and suggestions.Andrew SQLDBA

Query Help

Posted: 11 Sep 2013 09:06 AM PDT

Hi I want to update Flag column in second table based on the Adder names.If the Applicatiion has atleast one AIX and Adder name is UDB then the flag would be True.If the Application has more the one AIX and Adder names are diferent then the flag would be null.APpName OS Adder App1 ||| Windows|||NullApp1 ||| Linux |||UDBApp1 ||| AIX |||UDBApp1 ||| Linux |||SqlApp2 ||| AIX ||| UDBApp2 ||| Windows||| UDBApp2 ||| Linux ||| UDBApp2 ||| AIX ||| UDBOUTPUT SHOULD BE LOOK LIKE BELOWAPpName OS Adder FlagApp1||| Windows|||Null|||null App1||| Linux |||UDB |||null App1||| AIX |||UDB |||null App1||| Linux |||Sql |||null App2|||AIX ||| UDB|||TRUEApp2|||Windows||| UDB|||TRUEApp2|||Linux ||| UDB|||TRUEApp2|||AIX ||| UDB|||TRUELet me know fi you need addiitional information.ThanksAswin

Reset Identity Column only on 1 Times when Month have changed

Posted: 11 Sep 2013 06:15 PM PDT

Hi,I want to reset an Identity Column 1 Time in Month (by T-SQL -not job).How can I make sure that this action is performed only 1 Times every Month.Like this...if datepart(month,getdate()) != datepart(month,dateFromTableValue) Truncate Table DBCC CHECKIDENT('dbo.MyTable', RESEED, 1); Regards Nicole;-)

The working of PIVOT

Posted: 11 Sep 2013 08:12 PM PDT

Can you please explain how PIVOT is used in this query? i.e. logic behind the working of PIVOT in the query.SELECT * FROM ( SELECT IsNull(St.alternateStudyCode, St.studyCode) AS StudyCode, St.clientStudyReference as SponsorStudyCode, Cl.clientName AS Sponsor, SAFV.txtValue, SAF.extractName FROM Activities AS Ac INNER JOIN Studies AS St ON (St.studyIncId = Ac.studyIncId AND St.studySqlId = Ac.studySqlId) AND St.isDeleted = 0x0 AND St.studyStartDate < DateAdd(Day,1,@ToDate) -- study initiation date <= the end date AND (St.studyEndDate is null or St.studyEndDate >= @FromDate) -- study completion date >= the start date LEFT JOIN StudiesClients as StCl ON StCl.StudySqlId = St.studySqlId AND StCl.studyIncId = St.StudyIncId AND StCl.isDeleted = 0x0 AND StCl.sponsorRanking = 1 LEFT JOIN Clients AS Cl ON Cl.clientSqlId = StCl.clientSqlId AND Cl.clientIncId = StCl.clientIncId AND StCl.isDeleted = 0x0 LEFT JOIN StudiesCategories as StCa ON StCa.studyCategorySqlId = St.studyCategorySqlId AND StCa.studyCategoryIncId = St.studyCategoryIncId AND StCa.isDeleted = 0x0 LEFT JOIN StudiesStatuses AS StSt ON StSt.studyStatusSqlId = St.studyStatusSqlId AND StSt.studyStatusIncId = St.studyStatusIncId AND StSt.isDeleted = 0x0 LEFT JOIN StudiesAdditionalFieldsValues As SAFV LEFT JOIN StudiesAdditionalFields AS SAF ON SAF.studyAdditionalFieldSqlId = SAFV.studyAdditionalFieldSqlId AND SAF.studyAdditionalFieldIncId = SAFV.studyAdditionalFieldIncId AND SAF.isDeleted = 0x0 --Study link to Site LEFT JOIN CboValues AS StCbo ON SAFV.cboRecordSqlId = StCbo.cboValueSqlId AND SAFV.cboRecordIncId = StCbo.cboValueIncId AND StCbo.isDeleted = 0x0 ON SAFV.studySqlId = St.studySqlId AND SAFV.studyIncId = St.studyIncId AND SAFV.isDeleted = 0x0 AND SAF.extractName = 'GLPSite' LEFT JOIN Operators AS Op ON Op.operatorSqlId = Ac.todoBySqlId AND Op.operatorIncId = Ac.todoByIncId AND Op.isDeleted = 0x0 INNER JOIN TypesOfActivities AS TypOA WITH(NOLOCK) ON TypOA.typeOfActivitySqlId=Ac.typeOfActivitySqlId AND TypOA.typeOfActivityIncId=Ac.typeOfActivityIncId AND TypOA.isDeleted=0x0 AND TypOA.typeOfActivityCode = 'EAS-1' WHERE Ac.isDeleted = 0x0 AND ( (@siteSqlId = StCbo.cboValueSqlId AND @siteIncId = StCbo.cboValueIncId) OR (@siteSqlId = AcCbo.cboValueSqlId AND @siteIncId = AcCbo.cboValueIncId) )) As SourceTablePIVOT ( Min(TxtValue) FOR extractName In ( [TestItem], [TestSystem], [Crop], [Organism], [QASystem] ) ) As PivotTable

Only functions and some extended stored procedures can be executed from within a function.

Posted: 11 Sep 2013 07:25 AM PDT

Hi,I have this function called fn_GetTimedKey() when I run it in my SQL2005 environment it works fine.When I run it in my SQL2008R2 environment I get the error:Only functions and some extended stored procedures can be executed from within a function.The SQL2008R2 server is new. What can I look for?Here's the code for the function:[code="sql"]BEGIN DECLARE @v_Key CHAR(12) EXEC master..xp_usg_gettimedkey_VAL @v_Key OUTPUT RETURN @v_KeyEND[/code]

SELECT TOP ????

Posted: 11 Sep 2013 04:46 AM PDT

Hello EveryoneI am a bit curious after doing some accidental testing. I was going thru someone elses code, and noticed that on a few Select queries, they were using SELECT TOP 50000 etc, etc....But there are only something like 60 rows in that table. I removed the TOP clause and the query ran much faster and with a lot less CPU cycles. Is there truly something less efficient about using the TOP clause, with some really high number, when there are not nearly that many rows in the table? That is almost like using SELECT TOP 100% without a WHERE clause.Does removing the TOP clause and the large number, really increase the performance when there are not nearly that many rows in that particular table?Thanks in advance for any and all suggestions, comments and adviceAndrew SQLDBA

Trapping error from sp in another db

Posted: 11 Sep 2013 05:58 AM PDT

SQL Server 2008 R2Two computers: MachineOne and MachineTwoTwo instances of SQL server,: MachineOne.ServerOne and MachineTwo.ServerTwoTwo databases:MachineOne.ServerOne.DBOne, and Machine2.Server2.DBTwo.On [MachineTwo].[ServerTwo].[DBTwo] I write an sp that calls an sp from [MachineOne].[ServerOne].[DBOne][code]EXEC [MachineOne].[ServerOne].[DBOne].[dbo].[MySP][/code]On ServerOne [MySP] does a bunch of validation and uses RAISEERROR to break when something fails. The code there that raises the flag looks like[code] SET @i_ReturnMessage = dbo.udf_SystemMessage(100000,1)RAISERROR (@i_ReturnMessage, 16, 1)RETURN @@ERROR[/code]When everything is right as expected it works fine. I want to trap an error in MySP, though, so on ServerTwo I call[code]BEGIN TRY EXEC @ErrorReturn = [MachineOne].[ServerOne].[DBOne].[dbo].[MySP]END TRYBEGIN CATCH @ErrorMessage = ERROR_MESSAGE(), @ErrorSeverity = ERROR_SEVERITY(), @ErrorState = ERROR_STATE(), @ErrorLocation = 'DoExistingEmployeeNewPasses', @ErrorDescription = @ErrorReturn+' - Adding Address Profile for '+@WFirstName+' '+@wLastName; EXEC [dbo].[LogError] @ErrorMessage, @ErrorSeverity, @ErrorLocation, @ErrorDescription ;END CATCH[/code]Where [LogError] is an sp on ServerTwo that should write the error to a log file.When I run it from SSMS connected to ServerTwo I get an error message as expected, but my TRY clause never seems to get triggered.How can I collect the error information on ServerTwo for logging? [MySP] on ServerOne is vendor code, so I can't do anything there to change how the flag is raised.

DATEPART WITH YEAR AND MONTH

Posted: 11 Sep 2013 06:54 AM PDT

Hi all how do I query datepart to include month also when passing a variable from my app. My attempt below WHERE (DATEPART(yyyy, @date1) = DATEPART(yyyy, Docket_Date) AND (DATEPART(m, @date1) = DATEPART(m, Docket_Date)))

Trapping error from sp in another db

Posted: 11 Sep 2013 05:46 AM PDT

SQL Server 2008 R2Two computers: MachineOne and MachineTwoTwo instances of SQL server,: MachineOne.ServerOne and MachineTwo.ServerTwoTwo databases:MachineOne.ServerOne.DBOne, and Machine2.Server2.DBTwo.On [MachineTwo].[ServerTwo].[DBTwo] I write an sp that calls an sp from [MachineOne].[ServerOne].[DBOne]EXEC [MachineOne].[ServerOne].[DBOne].[dbo].[MySP][MySP] does a bunch of validation and uses RAISEERROR to break when something fails.When everything is right as expected it works fine. I want to trap an error in MySP, though, so on ServerOne I callBEGIN TRY EXEC [MachineOne].[ServerOne].[DBOne].[dbo].[MySP]END TRYBEGIN CATCH

SP_SEND_DB_MAIL Alignment Issue

Posted: 11 Sep 2013 04:06 AM PDT

HiI have a script, which when ran inserts values from multiple tables into another table, then executes 'sp_send_dbmail'. I use the '@query' parameter in 'sp_send_dbmail' to select from the new table, so that the data that has been amalgamated into the new table is shown in the output when the email is sent. All this works fine.The issue I am having is to do with how the data is displayed in the email received, the columns and the corresponding data do not align correctly, it's readable but obviously doesn't look great.For example, The corresponding data for Column A will not align correctly with Column A's heading, Column B's data will not align with Column B's heading etc.Hope this is clear, can anyone suggest a resolution? It's driving me crazy!

[SQL Server 2008 issues] Select several subqueries with more than 1 value

[SQL Server 2008 issues] Select several subqueries with more than 1 value


Select several subqueries with more than 1 value

Posted: 11 Sep 2013 06:57 PM PDT

Hi. There is no problem with this query[code="sql"]select (select 'hello'), (select 'world')[/code]But if one select statement return more than 1 value it throw exception.Initialization:[code="sql"]Create table table1(c nvarchar(1));Create table table2(c nvarchar(1));insert into table1 ('1')insert into table1 ('2')insert into table1 ('3')insert into table2 ('a')insert into table2 ('b')[/code]and here is query that generate error:[code="sql"]select (select c from table1), (select c from table2)[/code]I expect output be something like this:c,c------1,a2,b3,nullIt seems impossible, but I want to be sure if there is any trick for it or not.Thank you for help.

String as a date format

Posted: 11 Sep 2013 06:05 PM PDT

Hi All,From other source I am getting a data as string format like 'Sep 6 2013'. I want to see it as a Date format.and the date format should be DD/MM/YYYY. It means 'Sep 6 2013' looks like 06/09/2013.Is it possible to to that in SQL server?Please help!!

Why is my logon trigger blocking logons?

Posted: 11 Sep 2013 08:14 AM PDT

I've created the following trigger to track logons:[code="other"]Use masterGoCreate database AuditDBGoUSE [AuditDb]GO CREATE TABLE [dbo].[ServerLogonHistory]( [SystemUser] [varchar](512) NULL, [DBUser] [varchar](512) NULL, [APP_NAME] [varchar](512) NULL, [SPID] [int] NULL, [LogonTime] [datetime] NULL, [HOST_NAME] [varchar](512) NULL) ON [PRIMARY] GO Grant insert, update, select on ServerLogonHistory to publicGo CREATE TRIGGER [Tr_ServerLogon]ON ALL SERVER FOR LOGONASBEGINif( (ORIGINAL_LOGIN() <> 'sa')and APP_NAME() not like 'SQLAgent%'and APP_NAME() not like '%IntelliSense')INSERT INTO AuditDb.dbo.ServerLogonHistorySELECT SYSTEM_USER,USER,APP_NAME(),@@SPID,GETDATE(),HOST_NAME()END GO[/code]When I run this all is good, table is in place, etc. but I tested it by logging in as another user and I'm unable to logon. I then granted permission to the table explicitly to that user and am still unable to logon. When I check the user it is a member of public. When I check the table under permissions-effective tab I get the following for public: "[i]cannot execute as the database principal because the principal "public" does not exist, this type of principal cannot be impersonated, or you do not have permission[/i]"For the other user it says: "[i]the server principal "domain\username" is not able to access the database "AuditDb" under the current security context[/i]"I'm missing something here - but am stumped as to what it is. Can anyone point out the obvious to me? :-)Oracle is my usual forte, and implementing a logon trigger is a little simpler process.

Max Memory

Posted: 11 Sep 2013 10:12 AM PDT

I have set Max Memory to 6 GB. But SQL Server is using only 2.9 GBQuery used:SELECT object_name, counter_name, cntr_value AS 'Total Server Memory (KB)'FROM sys.dm_os_performance_counters WHERE counter_name = 'Total Server Memory (kB)'So why SQL Server is not using all allocated Max memory i.e 6GB? Is this normal? SQL Server should use all allocated Max memory right?Thanks,Gary

Strange Working of Sql server database mail- Need urgent help

Posted: 11 Sep 2013 04:48 AM PDT

It may be a stupid question/problem,,, But i am struck up in a situation where i am not able to send emails from database email.After much surfing on internet to find some solution, i came cross a script where it checks for the DB mail service status, if it is stopped, then again a script to start the service.It is as Follows:-- See if database mail (DM) is startedEXEC msdb.dbo.sysmail_help_status_sp ------- This Gives Output of STOPPEDGO-- Stop the serviceEXEC msdb.dbo.sysmail_stop_sp ---- This runs successfully.GO-- See if database mail (DM) is stoppedEXEC msdb.dbo.sysmail_help_status_sp ----- This Gives STOPPED (as expected)GO-- Start the serviceEXEC msdb.dbo.sysmail_start_sp ---- This runs successfully.GO-- See if database mail (DM) is startedEXEC msdb.dbo.sysmail_help_status_sp ---- Now this executed just after the previous Query and give the output as STARTED, but after few seconds when i again execute this. output goes back to STOPPEDGOWhy does it STOPPED automatically when once it is started.

Getting duplicate values evenif used distinct

Posted: 11 Sep 2013 12:47 AM PDT

Hi All,I used distinct in my query but then also I am getting almost 100K of duplicate records, below is the detail of count.Expected count : 1769894From select query I am getting : 13883717The contact_ID should be unique and we shouldnt have duplicate contact ID, I used 4-5 tables to join.Below is the query,drop table CnI_Test;create table CnI_Test asselect distinctml.account_id, ml.account_name, ml.NAMED_ACCOUNT_IND, ml.NAMED_ACCOUNT_DESCRIPTION, ml.site_id, ml.org_name, ml.contact_id , ml.FIRST_NAME, ml.LAST_NAME,ML.JOB_AREA_CD, ML.JOB_AREA_CD_desc,ML.job_level_cd, ml.job_level_cd_desc,ML.MKT_REGION_NAME, ML.OVERALL_SALES_REGION, ML.SALES_REGION, ML.ISO_COUNTRY_CD_DESC, ML.TERRITORY,ml.CITY,ML.STATE_PROV_COUNTY,ml.MKTG_CAPABLE_FLAG, ML.MKTG_CAPABLE_EMAIL_FLAG, ml.MKTG_CAPABLE_PHONE_FLAG, ml.MKTG_CAPABLE_MAIL_FLAG, ML.LAST_DT_CONTACT_ACTIVITY,ML.JOB_TITLE,ml.AI_DATE_ADDED,CASE WHEN ML.MKTG_CAPABLE_PHONE_FLAG = 'Y' THEN ML.PHONE_COUNTRY_CD ELSE NULL END AS PHONE_COUNTRY_CD,CASE WHEN ML.MKTG_CAPABLE_PHONE_FLAG = 'Y' THEN ML.CONTACT_PHONE_NUMBER ELSE NULL END AS CONTACT_PHONE_NUMBER,CASE WHEN ML.MKTG_CAPABLE_PHONE_FLAG = 'Y' THEN ML.CONTACT_MOBILE_PHONE ELSE NULL END AS CONTACT_MOBILE_PHONE,CASE WHEN ML.MKTG_CAPABLE_PHONE_FLAG = 'Y' THEN ML.ORG_PHONE_NUMBER ELSE NULL END AS ORG_PHONE_NUMBER,CASE WHEN ML.MKTG_CAPABLE_EMAIL_FLAG = 'Y' THEN ML.EMAIL_ADDRESS ELSE NULL END AS EMAIL_ADDRESS,CASE WHEN ML.MKTG_CAPABLE_MAIL_FLAG = 'Y' THEN ML.ADDRESS_LINE_1 ELSE NULL END AS ADDRESS_LINE_1,CASE WHEN ML.MKTG_CAPABLE_MAIL_FLAG = 'Y' THEN ML.ADDRESS_LINE_2 ELSE NULL END AS ADDRESS_LINE_2,CASE WHEN ML.MKTG_CAPABLE_MAIL_FLAG = 'Y' THEN ML.ADDRESS_LINE_3 ELSE NULL END AS ADDRESS_LINE_3,CASE WHEN ML.MKTG_CAPABLE_MAIL_FLAG = 'Y' THEN ML.ADDRESS_LINE_4 ELSE NULL END AS ADDRESS_LINE_4,no_of_csu_job_areas.job_area as csu_job_areas,Owner_1.csu as CSU_DRIVER,SUBSTR(Owner_1.own,1,1) as CSU_Owner, SUBSTR(Owner_1.own,2,1) as CSU_Interest,Owner_2.NCV_DRIVER as NCV_DRIVER, SUBSTR(Owner_2.own,1,1) as NCV_Owner,SUBSTR(Owner_2.own,2,1) as NCV_Interest, NVL(ep.ACCOUNT_CURRENCY, 'Unmatched') AS ACCOUNT_SEGMENT ,NVL(g.now_in_gov,'No') AS now_in_gov ,NVL(g.release_from_gov_date,'N/A') AS release_from_gov_date,NVL(g.days_to_release_from_gov,'N/A') AS days_to_release_from_gov,sa.sales_req_not_to_call_accountfrom M_LIST MLleft join MRTROLLUP.CSU_JOB_AREAS no_of_csu_job_areas on (no_of_csu_job_areas.contact_id = ml.contact_id ) LEFT JOIN mrtcustomer.ent_parent ep ON ep.ACCOUNT_ID = ml.ACCOUNT_IDLEFT JOIN (SELECT DISTINCT upper(trim(email_address)) AS email_address FROM REPOR01.BLOCKED_EMAIL WHERE email_address IS NOT NULL) blocked ON blocked.email_address = upper(trim(ml.email_address))left join REPOR01.MART_CSU_CONTACT Owner_1 on Owner_1.contact_id = ml.contact_id left join REPOR01.MART_NCV_CONTACT Owner_2 on Owner_2.contact_id = ml.contact_id left join (select distinct email_address, case when segmentation_last_run_date >= current_date - 15 then 'Yes' else 'No' end as now_in_gov, case when segmentation_last_run_date >= current_date - 15 then to_char(segmentation_last_run_date + 15) else 'N/A' end as release_from_gov_date, case when segmentation_last_run_date >= current_date - 15 then to_char(15 - (to_date(to_char(current_date,'YYYYMMDD'),'YYYYMMDD') - to_date(to_char(segmentation_last_run_date,'YYYYMMDD'),'YYYYMMDD'))) else 'N/A' end as days_to_release_from_gov from (select trim(lower(email_address)) as email_address, max(segmentation_last_run_date) as segmentation_last_run_date from mrtaprimoadm.ca_main_comms_log where email_address is not null group by trim(lower(email_address)))) g on g.email_address = trim(lower(ml.email_address))left join sfdcdata.sfdc_accounts sa ON sa.enterprise_id = ml.account_idleft join sfdcdata.sfdc_accounts sa2 on ml.sfdc_account_id = sa2.sfdc_account_id;Can anyone please tell me how can I get expected 1769894 count.M_List table is having 1769894 contacts.Thanks,Abhi

optimize pivot query

Posted: 11 Sep 2013 01:00 AM PDT

Hi,I did this code to select 2 best selling prodID (by quantity) for each Year period and put it in pivoting format needed for report (ssrs)Just curious how it can be optimized (or left formatted inside of ssrs)this is my sample[code="other"];with t4 as (select '2011-01-02' as dd, 123 as prodID unionselect '2011-01-03' as dd, 345 as prodID unionselect '2011-01-04' as dd, 345 as prodID unionselect '2011-01-05' as dd, 123 as prodID unionselect '2011-01-06' as dd, 123 as prodID unionselect '2012-01-02' as dd, 567 as prodID unionselect '2012-01-03' as dd, 567 as prodID unionselect '2012-01-04' as dd, 567 as prodID union select '2012-01-04' as dd, 777 as prodID union select '2012-01-05' as dd, 777 as prodID union select '2012-01-06' as dd, 888 as prodID )--select * From t4select *, RANK () over (partition by year order by numb desc) as Rankinto #tt from (select COUNT(*) numb, DATEPART(year,dd) year, prodID from t4 group by DATEPART(year,dd), prodID ) b ---select * from #ttselect t1.prodID as best_11 , t2.prodID as best_12, t1.Rank from #tt t1 join #tt t2 on t2.rank = t1.rank where t1.year = '2011' and t2.year = '2012' and t1.Rank < 3 order by 3[/code]TxMario

sp_register_custom_scripting

Posted: 11 Sep 2013 02:02 PM PDT

Hi All,I have replication configured using a third party tool (power exchange). I get issues whenever i execute schema level changes. Do we have to register stored procedures that does only schema changes or do we have to register stored procedures that exectues DML changes as well?Please let me know.Thanks,

Calculate totals in each category by day

Posted: 11 Sep 2013 05:26 AM PDT

I need to calculates for each of a category by day, and I need to carry over those totals where a day isn't present. Just to clarify this is not a running total - This is "how many people were in __ status by date?"Here is some sample data:[code="sql"]CREATE TABLE #StCg (StDt DATE NOT NULL, StID VARCHAR(25) NOT NULL, NwStStg VARCHAR(75) NOT NULL)INSERT INTO #StCg(StDt, StID, NwStStg) VALUES('2013-08-01', 'a', 'New'),('2013-08-03', 'a', 'Call'),('2013-08-07', 'a', 'Start'),('2013-08-02', 'b', 'New'),('2013-08-07', 'b', 'Call'),('2013-08-04', 'c', 'New'),('2013-08-06', 'c', 'Canc')SELECT *FROM #StCg[/code]Here is the output with an extra column to explain things further. I put the output on an Excel sheet to make it easier to see.[img]https://dl.dropboxusercontent.com/u/7186315/SS.PNG[/img]

One sa password to rule them all?

Posted: 11 Sep 2013 12:38 AM PDT

Just wanting to get your views on the use of a common sa password for all instances vs individual sa passwords. Obviously the sa login is to only to be used in emergencies and the password will be really complex.What do you do?

SSIS PACKAGE DATA DUMP FROM SQL VIEW TO CSV AND PUT IT ON A FTP SERVER

Posted: 11 Sep 2013 05:54 AM PDT

Hi,I have a scenario where I need to create a SSIS Package and make it run automatically every night.What the package should do is that it should pull data from a sql view and dump it into a CSV file on the local server. The CSV file should have a date stamp appended to the file name.Now I have to dump the latest csv file from the local path on to the FTP Server.Can anyone give me step wise as to how this can be achieved?Example:I have csv files saydata_2103_09_08data_2013_09_09data_2013_09_10data_2013_09_11 on the local path c:\timesheetswhen the package is run on the night of 2013_09_11 the csv file data_2013_09_11 should be picked and be placed on the FTP Server.Thank You all in Advance

Disk io calculation issue .

Posted: 11 Sep 2013 05:51 AM PDT

I am trying to fetch the spid consuming highest disk io , I am getting the desired result from sysprocesses but I am not able to fetch the SQL text and SQL plan from that spid . Reason I am not able to is because I am using below approach to get the sql text and query plan which results all spids from history too . SELECT query_plan,session_id,text FROM sys.dm_exec_query_stats qs WITH (nolock)CROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) qtCROSS APPLY sys.dm_exec_query_plan(qs.plan_handle) qpCROSS APPLY sys.dm_exec_sessions siWHERE si.session_id = <SPID>Any one has other work around to find the SPID consuming highest diskio with SQL text and SQL plan.

Use data from Flat File for Variable in SSIS 2008

Posted: 11 Sep 2013 02:27 AM PDT

I am trying to use a text file to provide a variable in an SSIS package so that the end user can input different invoice numbers to be run through a process. I have found some help online which says to use a Script Component to pull the data from the file into the Variable however I have not written much Visual Basic code in the past and I have got a bit stuck. I am using Visual Studio 2008 however the article I have found online is using Visual Basic for Applications and I keep getting errors when I use that code. Help!

sys.dm_db_stats_properties WHERE modification_counter &lt; 0

Posted: 11 Sep 2013 03:43 AM PDT

Hi,In SQL Server 2008 R2 with SP2, we can get statistics that are [b]NOT modified[/b] using the query [i]SELECT * FROM sys.dm_db_stats_properties WHERE modification_counter < 0[/i]But I want find the same information in SQL server 2008 R2 SP1. Please advise what query I can use?Thanks,Gary

How to generate sequence of rows?

Posted: 11 Sep 2013 01:21 AM PDT

Hi Friends,I have a table having some data where i want to generate sequence of rows based on premise value. I have given here the samples and the results that i am looking for..please give me your suggestions if there is any T-SQL can do this. Any suggestions would be really appreciated. Friends, If i am not making you clear, please let me know, i can brief you. [code="sql"]-- My Dataselect '12th Main St' as StreetName, 1 as FromPremise , 5 as ToPremiseunionselect '10th Main St' as StreetName, 10 as FromPremise , 12 as ToPremise-- Expecting Resultselect '12th Main St' as StreetName, 1 as FromPremise , 1 as ToPremiseunionselect '12th Main St' as StreetName, 2 as FromPremise , 2 as ToPremiseunionselect '12th Main St' as StreetName, 3 as FromPremise , 3 as ToPremiseunionselect '12th Main St' as StreetName, 4 as FromPremise , 4 as ToPremiseunionselect '12th Main St' as StreetName, 5 as FromPremise , 5 as ToPremiseunionselect '10th Main St' as StreetName, 10 as FromPremise , 10 as ToPremiseunionselect '10th Main St' as StreetName, 11 as FromPremise , 11 as ToPremiseunionselect '10th Main St' as StreetName, 12 as FromPremise , 12 as ToPremise[/code]

Combine fields

Posted: 10 Sep 2013 11:23 PM PDT

Hello,I have to create a table and fill it with data from another tables.My question is can you held me with a script for a field in this table, which should be sum of the values of 7 other fields.Thanks,

Possibly Dumb Question

Posted: 11 Sep 2013 02:01 AM PDT

Why would you want to add SSAS or SSRS as a feature of a database engine instance as opposed to creating a new instance on the same box. What's the use cases?

set SET FMTONLY ON/OFF at database level

Posted: 11 Sep 2013 12:45 AM PDT

Developers are having an issue with BizTalk and SQL Server . They asked me if we could SET FMTONLY ON at the database level. Their are several 3rd party apps which run on this server and I have reservations about doing this

Total calculated in stored procedure for report sometimes returned as 0

Posted: 11 Sep 2013 01:22 AM PDT

This is a problem that I thought was a Reporting Services issue but now realize is something different, possibly in a SQL Server driver.We have some reports where we compute report totals for head count and various compliance percentages in the stored procedure. There have been intermittent problems where the head count incorrectly displays as 0 on the SSRS report. The stored procedures, executed in SQL Query Analyzer, always return the correct value. SSRS problem, right?Well...I was recently directed to begin porting our reports to a new web site using CakePHP. As an interim reporting solution we're using the existing stored procedures (the new website will have a new database but for some period of time the old & new databases will be synced). I'm finding that the same reports rendered to the browser using PHP intermittently display 0 for the head count on these same reports!Has anyone else ever run into this?

Backup Incomplete

Posted: 10 Sep 2013 09:36 PM PDT

Hi GuysWe have a VB application which includes SQL Express 2008R2. As part of that, we include an application which runs on the server and allows non-technical users to perform various admin tasks including backup. The problem is that occasionally the backup appears to be created but is in fact marked *** INCOMPLETE ***The sql that performs the backup is created like this: sql = "BACKUP DATABASE " & dbname & " TO DISK = '" & buPath & "' WITH COPY_ONLY"The strings are held in the application and not just typed in by the user.I have searched on here and via Google for clues as to why this happens but I can only find suggestions about what to do with an incomplete backup when you've got one, but nothing on what might cause it. Any suggestions much appreciated.Thanks

DBCC FREEPROCCACHE

Posted: 10 Sep 2013 07:45 PM PDT

What are the best practices around using DBCC FREEPROCCACHE to free up space in TEMPDB when it won't shrink?TEMPDB ran out of space on one of our systems, and therefore the query was cut short. It now won't shrink. sys.dm_exec_requests is showing that nothing is happening on tempdb, there are no locks either.It is a shared, production system so we do not want to restart the instance.

Difference

Posted: 10 Sep 2013 09:39 PM PDT

What is the difference between T-sql and sql server?

different IP address can connect SQL?

Posted: 10 Sep 2013 11:22 PM PDT

Hi,we have two IPs configured in server one is Primary IP and Backup IP..whenever install SQL Server by default taking & assign primary IP only also application ODBC configured.Is it possible for connect backup IP address for ODBC?can we do that thru configuration manager to remove the primary IP and assign backup IP, then restart the server?rgdsananda

SQL Server 2008 using more than Max Specified Memory

Posted: 25 Aug 2011 02:31 AM PDT

We have a newer SQL Server 2008 R2 machine running on Server 2008 x64, and we noticed that the sql service was eating up all the RAM on the box. I then applied a maximum server memory of 45056, and the service has since surpassed this number and is now consuming over 47 GB out of the 48 GB on the box. Any ideas as to why it would ignore the specified number?Thanks!

Format equivalant function in sql server 2008

Posted: 10 Sep 2013 07:43 PM PDT

Team,I have used format function sql server 2012 to get month number as "09"select format(dateadd(m,-11,'2013-08-01'), 'MM')Could you please help me to get same output in SQL server 2008.Thanks in advance,Keerthy

Wednesday, September 11, 2013

[how to] Not able to login in Oracle 11g console using sys/system@SID as sysdba

[how to] Not able to login in Oracle 11g console using sys/system@SID as sysdba


Not able to login in Oracle 11g console using sys/system@SID as sysdba

Posted: 11 Sep 2013 08:12 PM PDT

I am not able to login into Oracle 11g console using administrative account, when I tried it gives me error of wrong username and password. Then I tried from SQL plus, it got logged in without any problem. I use PL/SQL developer also , on connecting the database using it I am receiving error of insufficient privileges. Request you tell me the solution for this, as if SYS account is locked than SQL plus will also not work for it, which is not the case here.

How do I access the full text of an error in SSIS GUI?

Posted: 11 Sep 2013 07:11 PM PDT

I have an Excel task which is providing error text through the GUI, but I don't know how to access the error task without the GUI and the error disappears if I move the cursor from the X

enter image description here

I have the project property Run64BitRuntime set to False (though it is not editable so even if I wanted to, couldn't change it to true)

MySQL Table across multiple servers

Posted: 11 Sep 2013 02:58 PM PDT

I have been looking at different types of replication, and was wondering if this type is available:

Replication

To explain this more, here is how I am visioning this working:

The Setup

  • You have one master server that forwards queries to 2+ slaves
    • The slaves don't have all the data, for example if you have 2 slaves each would have 50% of the data. 3 slaves would have 33% of the data, and so on.
  • Slaves return data to the master
  • Master returns final data

The Process

  1. If a select was sent to the master
    1. The master would forward the select to the slaves
    2. The slaves would search their personal databases/tables
    3. The slaves would return their result set to the master
    4. The master would reassemble the slaves result sets into a final result set
    5. The master would return the result set to the web server/service
  2. If a INSERT/UPDATE/DELETE was sent to the master
    1. The master would forward the query to the slaves
    2. The slaves would INSERT/UPDATE/DELETE their personal data
    3. Once done they alert the master
    4. Once all slaves alert the master master alerts the web server/service

So, basically I would like to know, is this type of database setup possible with MySQL?

BerkeleyDB: Receiving truncated keys to bt_compare function in python BTREE

Posted: 11 Sep 2013 02:38 PM PDT

I am using BerkeleyDB 6.0 with bsddb3 python drivers. I have a dataset with BTREE access method having keys as strings representing floating point numbers. I have set a compare function to be used in set_bt_compare().

When I try to use db.set_range(key) function, the keys that the compare function receives are sometimes truncated. for instance,

--------------------------------------------------  'left :1378934633890000.0'  --------------------------------------------------  'right:13789346362'  

Here, the right key should be '1378934636286548.8'.

Has anyone seen this problem? have any suggestions as to how to fix it?

Thank you.

Postgres functions vs prepared queries

Posted: 11 Sep 2013 01:14 PM PDT

I'm sure it's there but I've not been able to find a simple definitive answer to this in the docs or via Google:

In Postgres, are prepared queries and user defined functions equivalent as a mechanism for guarding against SQL injection? Are there particular advantages in one approach over the other?

Thanks

Can I have a Distributed Database in which one database is MariaDB and the other one is MySQL Cluster?

Posted: 11 Sep 2013 12:33 PM PDT

This is a hypothetical question of course, I just want to know, because that's one of the question i'm going to be asked in my presentation and I have a feeling that the answer is "yes" after all my research in these past 2 weeks.

Seting timezone per database in MySQL

Posted: 11 Sep 2013 12:27 PM PDT

I wanted to know if MySQL let the admin set a timezone per database ?

I have three different databases for three different customers. Each of them being in their own timezone.

I would like to load the exact same event for each customer and would like it to run everyday at 00:00:01 for their respective timezone. Is it possible ?

I can load three different sql script specifying in each their respective timezone, but it doesn't scale well and it make me have three version of almost the same file.

SQL output formatting

Posted: 11 Sep 2013 04:38 PM PDT

I want this:

"ACCOUNT_ID","MAJOR_VERSION","MINOR_VERSION","COUNTRY","ACCEPTANCE_TIMESTAMP","AGREEMENT_ID"  "abcdefgh-1234-5678-ijkl-mnopqrstuvwx","20110901","1","CN","1329574013737","tos"  

But I get this:

'"'||"ACCOUNT_ID"||'"'||','||'"'||"MAJOR_VERSION"||'"'||','||'"'||"MINOR_VERSION"||'"'||','||'"'||"COUNTRY"||'"'||','||'"'||"ACCEPTANCE_TIMESTAMP"||'"'||','||'"'||"AGREEMENT_ID"||'"'  "abcdefgh-1234-5678-ijkl-mnopqrstuvwx","20110901","1","CN","1329574013737","tos"  

Using the following SQL:

select '"'|| "ACCOUNT_ID"||'"'||','||    '"'|| "MAJOR_VERSION"||'"'||','||    '"'|| "MINOR_VERSION"||'"'||','||    '"'|| "COUNTRY"||'"'||','||    '"'|| "ACCEPTANCE_TIMESTAMP"||'"'||','||    '"'|| "AGREEMENT_ID"||'"'  from THE_TABLE aaa  where aaa.country='CN' and rownum < 10;  

How can I get the first output ?

Under what conditions does SQL Server encrypt tempdb?

Posted: 11 Sep 2013 12:37 PM PDT

When does tempdb get encrypted with Transparent Data Encryption? What configuration, etc, causes tempdb to be encrypted this way?

Is it possible to dynamically retrieve the number of columns in a view in Oracle?

Posted: 11 Sep 2013 12:42 PM PDT

As the title says, is it possible to retrieve the number of columns in a view dynamically?

I'm learning Oracle 11gR2 and I'm checking out the V$ views. One of the things I want to do is output them to file (for various reasons I am unable to spool the document) so I was going to output using java, however I need to know how many columns are in each view - a daunting task with 536 views, and now I'm curious to if I can do this dynamically.

I tried using user_tab_columns but it returns 0 for views.

Note: performance is not key, this is a learning exercise for me, so doing it correctly is more important than doing it quickly (if its even possible)

One Materialized Views in Two Refresh Groups

Posted: 11 Sep 2013 08:37 PM PDT

I have 5 MViews that I want to refresh in two occassions, every sundays and at the 1st of the month. I created a Refresh Group for the weekly and that work fine. But when I tried to created the second Refresh Group for the monthly I get a "materialized view is already in a refresh group".

You can only have a materialized view in one refresh group? What options to I have to refresh it in different intervals?

Thanks

No SA password. No SQL Server Management Studio. No OS authentication

Posted: 11 Sep 2013 11:06 AM PDT

I have this problem.

I need to do some administrative tasks on a MS SQL Database using SA account.

  • OS authentication is not set.
  • SQL Server Manager Manager Studio is not installed.
  • I have no other admin database account.
  • I do have access to a Windows admin account
  • The only tools installed on the database server are as shown:

enter image description here

How can I activate OS authentication so I can log into the database an reset the SA password ?

Moving Data between Partitions

Posted: 11 Sep 2013 12:20 PM PDT

I have partition on my database up to 2013 and this partition contains >= 2013 records. Now I have test table which have more than 2013 records so I have created 2014 File and filegroup and add partition on quarterly basis. but as soon as partition created some records switched from 2013 partition to 2014 partitions.

I have tried with only one table with almost 12 million records,and I have almost 100 tables with more than 100 million records.so is it the good practise? or I have to do with the http://technet.microsoft.com/en-us/library/ms191174(v=sql.105).aspx approach.

Create user for alternate port

Posted: 11 Sep 2013 11:32 AM PDT

I am running a private MySQL instance on WebFaction

WebFaction is a shared host who provides both for their public MySQL instance and for your own private MySQL instance. The private instance is a 1-click install using their system which then provides a root user and port to you. The public instance is always just localhost, and the private instance is localhost:port.

What is the proper way to set up a user for that private instance?

CREATE USER 'user'@'localhost:port' IDENTIFIED BY 'password';
CREATE USER 'user'@'localhost' IDENTIFIED BY 'password';
CREATE USER 'user'@'%' IDENTIFIED BY 'password';

Does it matter?

I'm running into an issue trying to set up a Simplified Machines forum instance. The SMF forum will connect to localhost just fine, and it seems to connect to localhost:port okay (at the point in code where it makes the connection to the DB, it is successful), but then will only read and write to localhost and not localhost:port.

I have successfully created a db user which can connect to, read, and write via localhost:port, but refuses to do so via SMF.

Any thoughts?

Attach trigger to stored procedure

Posted: 11 Sep 2013 01:01 PM PDT

I have a CLR sp in SQL Server 2008 R2, and I want to count how often it is called, in order to create a statistic on data quality. (The sp allows manual correction of data).

How can I let a counter go up everytime the CLR sp is called? Do I necessarily have to change the sp itself?

Suggestions are appreciated.

Find the size of each index in a MySQL table

Posted: 11 Sep 2013 08:13 PM PDT

I am trying to figure out size of each index of a table.

SHOW TABLE STATUS gives "Index_length" is the summation of all the indices of the table. However if a table has multiple indices (e.g. in an employee table) then emp_id, ename and deptno are 3 different indices for which I want sizes separately.

emp_id : xx Kb    ename  : yy Kb    deptno : zz Kb     

How can I get these?

Can't connect to service after updating ODBC driver

Posted: 11 Sep 2013 08:22 PM PDT

I am upgrading a program at work and one of the changes is that it now uses PostgreSQL 9.2.4 instead of 8. I was getting a 'client encoding mismatch' error, so I updated the ODBC driver, and the problem went away. However, with the new driver, my program does not want to connect to a custom service that it uses anymore.

The custom service uses postgres a lot. The error I'm getting is '(10061) connection is forcefully rejected'. Postgres is configured to accept connections from any IP address, so I'm not sure why I'm getting this error. The program will connect fine to the custom service with the old version of the ODBC driver, but as soon as I start using the new driver, it does not want to connect. I've checked the services list and both postgres and the custom service are started.

At one point, while trying to connect to the custom service, I was getting an error that said something like "OLE DB error: cannot send query to the backend". However, I can't seem to reproduce this error message anymore, it is simply not connecting.

I don't have a lot of database experience, so I apologize if this information is confusing or incomplete. Please let me know if you need clarification on anything.

Any suggestions would be appreciated, even if they are just ideas on how to troubleshoot this issue.

Can I run concurrent backups of multiple read only filegroups?

Posted: 11 Sep 2013 06:10 PM PDT

I have 62 readonly filegroups in a SQL server 2008 enterprise database. Can I backup multiple read only filegroups at the same time? I would assume this to be the case but given this database is over 20TB in size I do not want to invalidate any files by giving it the ol' college try.

MySQL MyISAM index causes query to match no rows; indexes disabled, rows match

Posted: 11 Sep 2013 07:20 PM PDT

I created a table and index as described in this SE post, but when I query the table for a particular ID, no matches are found. When I disable the index, matches are found.

Commands ran:

CREATE TABLE mytable (id1 int, id2 int, score float) ENGINE=MyISAM;  LOAD DATA INFILE '50-billion-records.txt' INTO mytable (id1, id2, score);  ALTER TABLE mytable ADD INDEX id1_index (id1);    SELECT COUNT(*) FROM mytable; # returns 50 billion  SELECT COUNT(DISTINCT id1) FROM mytable; # returns 50K, should be about 50M  SELECT COUNT(*) FROM mytable WHERE id1 = 49302; # returns 0 results    ALTER TABLE mytable DISABLE KEYS;  SELECT * FROM mytable WHERE id1 = 49302 LIMIT 1; # returns 1 row  

Is this a bug with MySQL, or does this behavior make sense for some reason?

Note: When I ran ALTER TABLE mytable ENABLE KEYS; just now, the command is acting like it is building an index for the first time (it's still running after 30 minutes, and memory usage is at 80 GB, which matches my setting of myisam_sort_buffer_size=80G. I'll reply when this command finishes running (the original ALTER .. ADD INDEX.. took 7.5 hours, so it may be a bit).

Update: Running SHOW PROCESSLIST indicates "Repair with keycache" is taking place with my ENABLE KEYS command.

Update 2: I killed the repair job on the original index because after several hours, the memory and IO seemed pretty constant, and I hoped if I started over, it may just work. So in second pass, I rebuilt the table and index, and after doing so, the exact same result occurs.

As requested, here is explain for count queries with index enabled:

mysql> explain select * from mytable where id1 = 49302;  +----+-------------+-----------+------+---------------+-----------+---------+-------+------+-------------+  | id | select_type | table     | type | possible_keys | key       | key_len | ref   | rows | Extra       |  +----+-------------+-----------+------+---------------+-----------+---------+-------+------+-------------+  |  1 | SIMPLE      | mytable   | ref  | id1_index     | id1_index | 5       | const |    1 | Using where |  +----+-------------+-----------+------+---------------+-----------+---------+-------+------+-------------+  1 row in set (0.00 sec)    mysql> explain SELECT COUNT(DISTINCT id1) FROM mytable;  +----+-------------+-----------+-------+---------------+-----------+---------+------+-----------+--------------------------+  | id | select_type | table     | type  | possible_keys | key       | key_len | ref  | rows      | Extra                    |  +----+-------------+-----------+-------+---------------+-----------+---------+------+-----------+--------------------------+  |  1 | SIMPLE      | mytable   | range | NULL          | id1_index | 5       | NULL | 170331743 | Using index for group-by |  +----+-------------+-----------+-------+---------------+-----------+---------+------+-----------+--------------------------+  1 row in set (0.01 sec)  

Here is explains after disabling indexes (Note: 25 billion is correct number of records in table, not 50 billion as mentioned above):

mysql> explain select * from mytable where id1 = 66047071;  +----+-------------+-----------+------+---------------+------+---------+------+-------------+-------------+  | id | select_type | table     | type | possible_keys | key  | key_len | ref  | rows        | Extra       |  +----+-------------+-----------+------+---------------+------+---------+------+-------------+-------------+  |  1 | SIMPLE      | mytable   | ALL  | NULL          | NULL | NULL    | NULL | 25890424835 | Using where |  +----+-------------+-----------+------+---------------+------+---------+------+-------------+-------------+  1 row in set (0.00 sec)    mysql> explain SELECT COUNT(DISTINCT id1) FROM mytable;  +----+-------------+-----------+------+---------------+------+---------+------+-------------+-------+  | id | select_type | table     | type | possible_keys | key  | key_len | ref  | rows        | Extra |  +----+-------------+-----------+------+---------------+------+---------+------+-------------+-------+  |  1 | SIMPLE      | mytable   | ALL  | NULL          | NULL | NULL    | NULL | 25890424835 |       |  +----+-------------+-----------+------+---------------+------+---------+------+-------------+-------+  1 row in set (0.00 sec)  

Update 3: Still hoping to solve this oddity. Is there something I can do with myisamchk that might fix this? Since I completely repopulated the table and rebuilt the index (i.e. starting from scratch) and got the same result, I assume this was not just some freak occurrence, and that it is due to some internal limit I'm unaware of. On a side note, I've tried switching to Postgres for this dataset, but running into some other unrelated issues (that I'll leave to a different question), so fixing this index is still a top priority for me. Thanks!!

Update 4: Running CHECK TABLE currently. Will post back with updates as I have them

Error: "Storage Engine for the Table Doesn't Support Nullable Columns" (SequelPro)

Posted: 11 Sep 2013 02:20 PM PDT

I'm trying to load a very normal .csv file (that was created from Excel 2011 for Mac) into SequelPro (using MySQL) with my Mac -- and I've recently started getting this error consistently. Can anybody let me know what it is and how to fix it?

An error occurred while trying to add the new table 'wblist' by    CREATE TABLE `wblist` (  `FILE` VARCHAR(255),   `FIRSTNAME` VARCHAR(255),   `MIDDLE` VARCHAR(255),   `LASTNAME` VARCHAR(255),   `FULLNAME` VARCHAR(255),   `GENDER` VARCHAR(255),   `ADDRESS` VARCHAR(255),   `CITY` VARCHAR(255),   `STATE` VARCHAR(255),   `ZIP` VARCHAR(255),   `PHONE` BIGINT(11),   `UNIT` VARCHAR(255),   `JOB` VARCHAR(255),   `AREA` VARCHAR(255),   `TIME` VARCHAR(255),   `MAILINGADDRESS` VARCHAR(255),   `MAILINGCITY` VARCHAR(255),   `MAILINGSTATE` VARCHAR(255),   `MAILINGZIP` VARCHAR(255),   `ID` BIGINT(11),   `CONFIDENCE` VARCHAR(255),   `BIRTHDATE` VARCHAR(255),   `AGE` INT(11),   `RACE` VARCHAR(255),   `ETHNICITY` VARCHAR(255),   `RELIGION` VARCHAR(255),   `PARTY` VARCHAR(255),   `REGISTRATIONDATE` VARCHAR(255),   `VOTERSTATUS` VARCHAR(255),   `OtherPhone` VARCHAR(255),   `POSSIBLEADDRESS` VARCHAR(255),   `POSSIBLEMAILADDRESS` VARCHAR(255),   `RECID` VARCHAR(255)) ENGINE=CSV;    MySQL said: The storage engine for the table doesn't support nullable columns  

This is stopping me before I'm able to import the table. Thanks for the help!

optimizing MySQL for traffic analytics system

Posted: 11 Sep 2013 08:20 PM PDT

background :

I've developed a URL shortener system like Bitly with same features , so the system also tracks clickers info and represent as graphs to the person who has shorten the link as analytics data. currently I'm using MySQL and have a table to store click info with this schema:

visit_id (int)  ip (int)  date (datetime)  country  browser  device  os  referrer (varchar)  url_id (int)  //as foreign key to the shortened URL  

and for now , just the url_id field has index

The system should represent click analytics in the time periods the user wants, for example past hour, past 24 hours , the past month , ...

for example to generate graphs for past month , I do following queries:

SELECT all DAY(date) AS period, COUNT( * )                           FROM (                            SELECT *                           FROM visits                          WHERE url_id =  '$url_id'                          ) AS URL                          WHERE DATE > DATE_SUB( CURRENT_TIMESTAMP( ) , INTERVAL 1 MONTH )                           GROUP BY DAY( DATE )    //another query to display clicker browsers in this period  //another query to display clicker countries in this period  // ...  

issues:

  • for a shortened link with about 500,000 clicks , it takes about 3-4 seconds to calculate just the first query , so for total queries about 10-12 seconds which is terrible.
  • lots of memory and CPU is needed to run such queries

questions :

1- how to improve and optimize the structure , so the analytics of high traffic links will be shown in less than 1 second(like bitly and similar web apps) and with less usage of CPU and RAM ? should I make an index on the fields date, country, browser, device, os, referrer ? if yes , how to do that for the field date because I should group clicks some times by DAY(date), sometimes by HOUR(date), sometimes by MINUTE(date) and ...

2- is MySQL suitable for this application? assume at maximum my application should handle 100 million links and 10 billion clicks on them totally. Should I consider switching to an NoSQL solution for example?

3- if MySQL is ok , is my database design and table structure proper and well designed for my application needs? or you have better recommendations and suggestions?

UPDATE: I made an index on column referrer but it didn't help at all and also damaged the performance and I think that's because of the low cardinality of this column (also others) and the big resulting index size related to the RAM of my server.

I think making index on these columns would not help to solve my problem, my idea is about one of these:

1- if using MySQL, maybe generating statistics using background processing for high traffic links is better instead of calculating lively at the user request.

2- using some caching solution like memcached to help MySQL with high traffic links.

3- using a NoSQL such as MongoDB and solutions like Map-Reduce which I am poorly familiar with and haven't used ever.

what do you think?

Primary replica set server goes secondary after secondary fails

Posted: 11 Sep 2013 05:20 PM PDT

I have a 2 servers replica set that, after the secondary fails the primary goes into secondary mode while the secondary is in STARTUP2 (recovering). The problem with this is that I can't use the collection stored in that replica set freely, I'm getting errors trying to use the collection:

pymongo.errors.OperationFailure: database error: ReplicaSetMonitor no master found for set: rs2  

Sometimes if I restart the mongod instances, the server rs2-1 is the primary for a while, but after some time (while the secondary is recovering) I see this in the logs of rs2-1 (the primary):

Tue May  7 17:43:40.677 [rsHealthPoll] replSet member XXX.XXX.XXX.XXX:27017 is now in state DOWN  Tue May  7 17:43:40.677 [rsMgr] can't see a majority of the set, relinquishing primary  Tue May  7 17:43:40.682 [rsMgr] replSet relinquishing primary state  Tue May  7 17:43:40.682 [rsMgr] replSet SECONDARY  Tue May  7 17:43:40.682 [rsMgr] replSet closing client sockets after relinquishing primary  

Is there an easy way to make the primary keep being primary after the secondary fails? Am I doing something wrong?

Thanks in advance!

MySQL backup InnoDB

Posted: 11 Sep 2013 01:20 PM PDT

I have a VoIP server running 24x7. At low peak hour at lease 150+ users are connected. My server has MySQL running with InnoDB engine on Windows 2008 platform. I like to take at least 2 times full database backup without shutting down my service.

As per Peter Zaitsev - the founder of percona, mysqldump –single-transaction is not always good.

read here if you are interested

As I'm not a DBA, I like to know in my scenario, which would be best solution to take a database backup?

Thanks,

Strange characters in mysqlbinlog output

Posted: 11 Sep 2013 11:20 AM PDT

Has anyone experienced this? Data replicates fine but when output in mysqlbinlog there are hidden characters that break the input?

  • mysqlbinlog Ver 3.3 for Linux at x86_64
  • mysql 5.5.28 server

Thanks! Julie

Connecting to a SQL Server database from a Flash program

Posted: 11 Sep 2013 12:20 PM PDT

I currently have the ability to utilize Microsoft SQL Server 2012. I am developing a project with Adobe Flash Builder 4.7.

If I link my database with Adobe Flash Builder is there any additional steps I must take in order to make the database live, or as long as my computer is running will this database be accessible from any device that is utilizing it?

In other words is this a LAN only system or does it automatically make itself available for the programs I link to it?

Why Does the Transaction Log Keep Growing or Run Out of Space?

Posted: 11 Sep 2013 03:06 PM PDT

This one seems to be a common question in most forums and all over the web, it is asked here in many formats that typically sound like this:

In SQL Server -

  • What are some reasons the transaction log grows so large?
  • Why is my log file so big?
  • What are some ways to prevent this problem from occurring?
  • What do I do when I get myself on track with the underlying cause and want to put my transaction log file to a healthy size?

Does SQL manage foreign key constraints in terms of Insertion?

Posted: 11 Sep 2013 01:26 PM PDT

Say I have a table tbl1 with data A1 and A2, where A1 is the primary key. Then I have another table tbl2 with data A3 and A1, both together being the primary key, referencing A1 as a foreign key from tbl1.

Am I able to insert a tuple into tbl2 that has an A1 that is not in tbl1? Does SQL manage this for us as an error? Or what happens in this situation?

Why are queries parsed in such a way that disallows the use of column aliases in most clauses?

Posted: 11 Sep 2013 11:29 AM PDT

While trying to write a query, I found out (the hard way) that SQL Server parses WHEREs in a query long before parsing the SELECTs when executing a query.

The MSDN docs say that the general logical parsing order is such that SELECT is parsed nearly last (thus resulting in "no such object [alias]" errors when trying to use a column alias in other clauses). There was even a suggestion to allow for aliases to be used anywhere, which was shot down by the Microsoft team, citing ANSI standards compliance issues (which suggests that this behavior is part of the ANSI standard).

As a programmer (not a DBA), I found this behavior somewhat confusing, since it seems to me that it largely defeats the purpose of having column aliases (or, at the very least, column aliases could be made significantly more powerful if they were parsed earlier in the query execution), since the only place you can actually use the aliases is in ORDER BY. As a programmer, it seems like it's missing a huge opportunity for making queries more powerful, convenient, and DRY.

It looks like it's such a glaring issue that it stands to reason, then, that there are other reasons for deciding that column aliases shouldn't be allowed in anything other than SELECT and ORDER BY, but what are those reasons?

Sql Server 2008 x64 ODBC Linked Server to Oracle Not Working

Posted: 11 Sep 2013 02:01 PM PDT

I have an install of Sql Server 2008 x64, installed on two boxes. One is a Win 7 x64 workstation (Sql Server 2008 x64 SP2 developer edition), the other is a Windows Server 2003 x64 (Sql Server 2008 x64 SP2 Enterprise edition). I am trying to create a linked server to an external vendor's Oracle instance. I have installed the Oracle ODAC 11g 64 bit drivers using the full install (not the XCopy version).

The drivers appear to have all been installed correctly. I have created and updated my tnsnames.ora file using the correct IP, port, etc to the remote server and rebooted. Using the 64 ODBC admin tool, I am able to create the ODBC connection to the Oracle server and the "Test" button returns as successful using the alias in the tnsnames.ora file and the correct user ID and password.

I then go in to Sql Server 2008 and try to create my linked servers. I can create the OLE DB linked server and it connects successfully, I can list the tables/views in the catalog and do queries against them with only one significant problem. For tables with TIMESTAMP fields, a normal 4-part query throws fits. Looking around this appears to be a common problem with OLEDB linked servers from Sql Server to Oracle and using OPENQUERY is the most common workaround, which I do have working.

The ODBC connection, which is what our vendor recommends using to connect to them, is where I have big problems. I can create the linked server, which appears to be successfully created using the System DSN ODBC connection I created earlier. I can view the lists of tables/views in the catalog and they all appear to show up correctly. However, when I try to get data, it fails completely.

If I try to right click on a view name and select Script -> SELECT To.... I get a message that says that:

[tablename] contains no columns that can be selected or the current user does not have permissions on that object.

If I try to script out a SELECT manually and run it, as I know most of the column names, I get an error message:

The OLE DB provider "MSDASQL" for linked server [linked server name] returned an invalid column definition for table [table name].

The vendor states that the user ID (the same one in both cases) has the proper rights to the tables/views, which it appears to as the OLE DB connection mostly works. The Oracle server is 10g, but I don't know if it's 32 or 64 bit. Would that make a difference?

Right now I'm working on getting this working from the Win7 x64 workstation, but a short test on the 2003 server yielded the same results. If I have to, I guess I can make the OLEDB/OPENQUERY solution work. However it's not ideal or recommended by our vendor. Any ideas what I might be missing on getting the ODBC connection working?

MySQL - fastest way to ALTER TABLE for InnoDB

Posted: 11 Sep 2013 02:00 PM PDT

I have an InnoDB table that I want to alter. The table has ~80M rows, and quit a few indices.

I want to change the name of one of the columns and add a few more indices.

  • What is the fastest way to do it (assuming I could suffer even downtime - the server is an unused slave)?
  • Is a "plain" alter table, the fastest solution?

At this time, all I care about is speed :)

Search This Blog