Showing posts with label sqlserver. Show all posts
Showing posts with label sqlserver. Show all posts

Friday, March 30, 2012

J# using JDBC for SQL Server

code:

/* Establish a test connection to remote SQLServer in J# using JDBC */
package JSharpConnTest;
import java.sql.*;
public class JSharpConnTest
{
private static ResultSet rs;
private static Connection conn = null;
private static Statement stmt = null;
private static final String sSubprotocol = "jdbc:microsoft:sqlserver://";
private static final String sServerName = "CLUSTERTEST";
private static final String sPortNumber = "1433";
private static final String sDBName = "ClientLetterWorkRequests";
private static final String sUserName = "sa";
private static final String sPassword = "1111";
private static final String sSQLQuery = "SELECT * FROM CLDatabases";
private static final String sURL = sSubprotocol + sServerName +
":" + sPortNumber + ";
databaseName=" + sDBName + ";
";
public static void main(String[] args)
{
getConnection();
}
public static void getConnection()
{
System.out.println( "Connecting to.. " + sURL );
try
{
//Register the driver
// Microsoft SQL Server 2000 Driver for JDBC
// --problem locating driver?
Class.forName("com.microsoft.jdbc.sqlserver.SQLServerDriver");
// Pass connection URL
conn = DriverManager.getConnection(sURL, sUserName, sPassword);
// Query database
stmt = conn.createStatement();
rs = stmt.executeQuery(sSQLQuery);
// Test data retrieval
while (rs.next());
{
String colOne = rs.getString("DATABASEKEY");
String colTwo = rs.getString("SERVERTYPE");
System.out.println(colOne + " " + colTwo);
}
}
catch (java.lang.ClassNotFoundException ex)
{
System.err.println("\nClassNotFoundException: " + ex.getMessage());
}
catch (SQLException ex)
{
System.err.println("\nSQLException: " + ex.getMessage());
}
catch (Exception ex)
{
System.err.println("\nException: " + ex.getMessage());
}
finally
{
// Release resources
try
{
if (conn != null)
conn.close();
if (stmt != null)
stmt.close();
}
catch (Exception ex)
{
System.err.println("\nException: " + ex.getMessage());
}
}
} // end getConnection()
} // end JSharpConnTest


This statement:
Class.forName("com.microsoft.jdbc.sqlserver.SQLServerDriver");
Causes the following exception:
[java.lang.ClassNotFoundException]{
"com.microsoft.jdbc.sqlserver.SQLServerDriver"}java.lang.ClassNotFoundExcep
tion
I installed Microsoft SQL Server 2000 Driver for JDBC
I believe the problem is with locating the driver, unlike Java, J# doesn't
use classpath enviromental variable. I am not sure how to add the driver
reference in J#.Hi
Why use JDBC when ADO.NET can do everything for you?
Regards
--
Mike Epprecht, Microsoft SQL Server MVP
Zurich, Switzerland
IM: mike@.epprecht.net
MVP Program: http://www.microsoft.com/mvp
Blog: http://www.msmvps.com/epprecht/
"RobertStout" <
RobertStout@.discussions.microsoft.com>
wrote in message
news:4FB06586-4C07-4886-A9E5-444A85DA4F31@.microsoft.com...
>
code:

>
/* Establish a test connection to remote SQLServer in J# using JDBC */
>
>
package JSharpConnTest;
>
import java.sql.*;
>
>
public class JSharpConnTest
>
{
>
private static ResultSet rs;
>
private static Connection conn = null;
>
private static Statement stmt = null;
>
private static final String sSubprotocol = "jdbc:microsoft:sqlserver://";
>
private static final String sServerName = "CLUSTERTEST";
>
private static final String sPortNumber = "1433";
>
private static final String sDBName = "ClientLetterWorkRequests";
>
private static final String sUserName = "sa";
>
private static final String sPassword = "1111";
>
private static final String sSQLQuery = "SELECT * FROM CLDatabases";
>
private static final String sURL = sSubprotocol + sServerName +
>
":" + sPortNumber + ";
databaseName=" + sDBName + ";
";
>
>
public static void main(String[] args)
>
{
>
getConnection();
>
}
>
>
public static void getConnection()
>
{
>
System.out.println( "Connecting to.. " + sURL );
>
>
try
>
{
>
//Register the driver
>
// Microsoft SQL Server 2000 Driver for JDBC
>
// --problem locating driver?
>
Class.forName("com.microsoft.jdbc.sqlserver.SQLServerDriver");
>
>
// Pass connection URL
>
conn = DriverManager.getConnection(sURL, sUserName, sPassword);
>
>
// Query database
>
stmt = conn.createStatement();
>
rs = stmt.executeQuery(sSQLQuery);
>
>
// Test data retrieval
>
while (rs.next());
>
{
>
String colOne = rs.getString("DATABASEKEY");
>
String colTwo = rs.getString("SERVERTYPE");
>
System.out.println(colOne + " " + colTwo);
>
}
>
}
>
catch (java.lang.ClassNotFoundException ex)
>
{
>
System.err.println("\nClassNotFoundException: " + ex.getMessage());
>
}
>
catch (SQLException ex)
>
{
>
System.err.println("\nSQLException: " + ex.getMessage());
>
}
>
catch (Exception ex)
>
{
>
System.err.println("\nException: " + ex.getMessage());
>
}
>
finally
>
{
>
// Release resources
>
try
>
{
>
if (conn != null)
>
conn.close();
>
if (stmt != null)
>
stmt.close();
>
}
>
catch (Exception ex)
>
{
>
System.err.println("\nException: " + ex.getMessage());
>
}
>
}
>
>
>
} // end getConnection()
>
} // end JSharpConnTest
>


>
>
This statement:
>
Class.forName("com.microsoft.jdbc.sqlserver.SQLServerDriver");
>
>
Causes the following exception:
>

[java.lang.ClassNotFoundException]{
"com.microsoft.jdbc.sqlserver.SQLServerDr
iver"}java.lang.ClassNotFoundException
>
>
I installed Microsoft SQL Server 2000 Driver for JDBC
>
I believe the problem is with locating the driver, unlike Java, J# doesn't
>
use classpath enviromental variable. I am not sure how to add the driver
>
reference in J#.|||If I could use ADO.NET I would of been done with the entire app a long time
ago.
Sadly, I have to use JDBC.
Can you help?
Thanks
-Rob
I would much rather use ADO.NET but I can't, I have to use JDBC.
"Mike Epprecht (SQL MVP)" wrote:

> Hi
> Why use JDBC when ADO.NET can do everything for you?
> Regards
> --
> Mike Epprecht, Microsoft SQL Server MVP
> Zurich, Switzerland
> IM: mike@.epprecht.net
> MVP Program: http://www.microsoft.com/mvp
> Blog: http://www.msmvps.com/epprecht/
> "RobertStout" <RobertStout@.discussions.microsoft.com> wrote in message
> news:4FB06586-4C07-4886-A9E5-444A85DA4F31@.microsoft.com...
> [java.lang.ClassNotFoundException]{"com.microsoft.jdbc.sqlserver.SQLServerDr
> iver"}java.lang.ClassNotFoundException
>
>sql

Wednesday, March 28, 2012

Iterate through all Excel files and all their sheets

hello,

i need to transfer (migrate ) the data from xl sheet to sqlserver but actually the thing is if the source excel file has different sheets, in each sheet i have the data

and i need to move the entire data( all the data that is present in all sheets of the excel file) to a single table into sql server

like wise i have many xl files ( which have many sheets ) .

for eg:

excel file 1:

-> sheet 1

-> sheet 2

-> sheet 3

excel file 2:

-> sheet 1

-> sheet 2

-> sheet 3

excel file 3:

-> sheet 1

-> sheet 2

-> sheet 3

now i need to get the data from all of the files and i need to insert into a single table ( sql server) in ssis package

so plz help me by giving the solution asap.

thanks

B L Rao

hello ,

while i am trying to transfer the data from xl file to table in sql server by using ssis package it is giving error saying that primary key violation and cant insert duplicate value.

i understood that there is some duplicate data but can i find where that duplicate data exists i mean in which row ? because it contains thousands of records.

thanks and regards

B L Rao.

|||You may accomplish this by using 2 nested Loops: One fairly simple, a foreach loop to iterate through all excel files; a second one to iterate through each excel sheet. I am not sure how to implement the second one; perhaps if the number and name of the sheets is always the same you could built a list of values in a variable and then have the excel component to get the table name from a variable. Just an Idea, you would need to figure out the details.

Wednesday, March 21, 2012

Issues with performance of XQuery on SQL Server 2005

Hi folks,

we are executing the following Xquery on SQLserver 2005.

select
policy_xml.query('/Policy/PolicyApplication/Inuserer/InsurerID'),
policy_xml.query('/Policy/PolicyApplication/Insurer/AccountIdentifier'),
policy_xml.query('/Policy/PolicyApplication/Insurer/Type'),
policy_xml.query('/Policy/PolicyApplication/Insurer/HolderName')
from policyTable
where
policy_xml.exist('/Policy/PolicyApplication/Insurer/PolicyOwner/EntityID[.="E_1"]') = 1

Its taking 50 Secs to search from 10000 records{without indexes}

The table has 3 columns sno,policy_id,policy_xml.

We have primary index on policy_id field and 1 secondary index(path index) on the table.
When we enable the index the query takes 380 secs.

The size of loan_xml column is about 110 Kb for each row. We need to keep the indexes for some more complex update Xqueries. Is there a way out to improve the performance of the XQuery we are using? Please let us also know the reasons of decrease in performance using indexes on the table. Do indexs have any issues related to XQuery performance?

We have an urgent requirement to resolve this issue. Kindly let us know the resolution ASAP.

Please let us know if you require any other information in this regard.

Thanks,

Bhuvanesh

I am not a XML guru to exactly know where the problem might be but want to pass the following link

which talks of some performance techiniques.

http://msdn2.microsoft.com/en-us/library/ms345118(SQL.90).aspx

Regards

AK

|||

Hi Bhuvanesh,

From what you said here:

The table has 3 columns sno,policy_id,policy_xml.

We have primary index on policy_id field and 1 secondary index(path index) on the table.

I assume you haven't create XML index on column policy_xml, the DDL statement will look like this

Code Snippet

createprimaryxmlindex p_xml_idx

on policyTable(policy_xml)

go

It should definitely improve your query performance. Please let me know if otherwise.

|||

I've similar issue.

Table1 ( Xid , XML_data (xml)) where Xid is primary key and XML_data is of xml datatype.

I've set primary index on XML_data along with the three secondary indexes.

with fillfactor 90, padindex on and sort _in_tempdb is on.

xml_data stores 1000 xml's of size 150 kb each. i'm fetching single xml for particular value and it is taking 20 minutes on server with 2 gb ram.

the query is


SELECT
xml_data.query('/Info/PersonalInfo/Entity[1]/LastName/text()'),
xml_data.query('/Info/PersonalInfo/Entity[1]/FirstName/text()'),
xml_data.query('/Info/PersonalInfo/Entity[1]/Sno/text()'),

xml_data.query('/Info/PersonalInfo/Entity[2]/LastName/text()'),
xml_data.query('/Info/PersonalInfo/Entity[2]/FirstName/text()'),
xml_data.query('/Info/PersonalInfo/Entity[2]/Sno/text()')
FROM Table1

Looking out for valuable help.....

|||

Try following query to see if any improvement. I change query() to value() since you seems want to get scalar value, not xml.

Code Snippet

SELECT

x.value('(.[1]/LastName)[1]','varchar(100)'),

x.value('(.[1]/FirstName)[1]','varchar(100)'),

x.value('(.[1]/Sno)[1]','varchar(100)'),

x.value('(.[2]/LastName)[1]','varchar(100)'),

x.value('(.[2]/FirstName)[1]','varchar(100)'),

x.value('(.[2]/Sno)[1]','varchar(100)')

FROM Table1 crossapply xml_data.nodes('/Info/PersonalInfo/Entity')as t(x)

issues with full text search (especially views and pick lists)

Hi, I am trying to implement a global full text search on our SQL
Server. Our app has several entities that are stored in the DB. I would
like to be able to search for 'John Doe' and get results in all types
of entities. Problem is:
-FT Search does not crawl views. Unfortunately, each entity type in
our system is not stored in full in one table. This is because we are
using pick lists and look ups. For instance: Industry type is stored as
a code that represents an entry in an industries table. However, all
these values are joined in a view to create all the required fields for
the entity a full text search query and index are created per table.
What would be an effective way to work around this problem? (I don't
see any solution to this problem in Yukon either.)
-Ranking: both 2000 and Yukon do not allow for merging of rankings
from several tables/queries. However, it is important to us to be able
to display results from several tables and sorted in a logical way. Any
suggestions/work arounds?
-Performance and scalability: How many rows and/or how many GB can I
have in my DB and still get good FT search query performance (less than
5 seconds)? (I need data for both 2000 and Yukon)
-Same with regards to indexing: ideally we would like to keep the
index up to date in real time. We would like to use the track changes
feature. However, our app allows many users to be logged in and
edit/delete/add entries in the DB. What is the maximum amount of DB
transactions per minute (second?) that will still allow us to keep the
index updated in real time? (I need data for both 2000 and Yukon)
-
I would greatly appreciate any suggestions/tips/info/workaround.
Thanks!
baraka,
Unfortunately, SQL Server 2000 FTS does not directly support the FT Indexing
of views. However, you can include FTS queries *in* views, but not create FT
Indexes *on* views in SQL Server 2000.
Since you're testing Yukon (SQL Server 2005), have you downloaded and
installed the June CTP version? If not, I'd highly recommend that you do so
and then create one of your views as an "indexed view" as FT Indexing an
indexed view is supported in SQL Server 2005.
Relative to merging ranking for multiple tables (in SQL 2000 or SQL 2005),
try the following example to dump them into a
temp table, the below option as a few advantages, especially as it relates
to "summing" via the MAX function the RANK value as well as using a group
by:
CREATE TABLE #FTSQueryResult (PK_ID int, Rank int)
INSERT #FTSQueryResult
SELECT PK, FTSTable.Rank FROM . . . (and the rest of your 1st query)
INSERT #FTSQueryResult
SELECT PK, FTSTable.Rank FROM . . . (and the rest of your 2nd query)
Select PK_ID, Max(Rank) as Rank From #FTSQueryResults
Group by PK_ID
In regards to performance and scalability, for SQL 2000 its less an issue
with amount of text, and more a factor of the number of rows and the
language of the text in the FT-enable columns as well as your server's
configuration - number & speed of procs, amount of RAM and speed of your
disk drives. Review SQL 2000 BOL title "Full-text Search Recommendations"
for more info! As for SQL 2005, I've worked with a customer (now in Yukon
TAP) who FT-indexed a table with 180 Million rows in 8 hours, while using
SQL 2000 on the same server and same database and FT-enable table took over
6 weeks to complete! Checkout the case study of an internal MS application
"SQL Server 2005 - 7x faster than SQL 2000 Full-Text Indexing" at
http://spaces.msn.com/members/jtkane/Blog/cns!1pWDBCiDX1uvH5ATJmNCVLPQ!433.entry
I'd highly recommend that you review all of the links at "SQL Server 2000
Full-Text Search Resources and Links"
http://spaces.msn.com/members/jtkane/Blog/cns!1pWDBCiDX1uvH5ATJmNCVLPQ!305.entry
Regards,
John
SQL Full Text Search Blog
http://spaces.msn.com/members/jtkane/
"baraka" <barak.benezer@.gmail.com> wrote in message
news:1122341762.314188.223230@.g14g2000cwa.googlegr oups.com...
> Hi, I am trying to implement a global full text search on our SQL
> Server. Our app has several entities that are stored in the DB. I would
> like to be able to search for 'John Doe' and get results in all types
> of entities. Problem is:
> - FT Search does not crawl views. Unfortunately, each entity type in
> our system is not stored in full in one table. This is because we are
> using pick lists and look ups. For instance: Industry type is stored as
> a code that represents an entry in an industries table. However, all
> these values are joined in a view to create all the required fields for
> the entity a full text search query and index are created per table.
> What would be an effective way to work around this problem? (I don't
> see any solution to this problem in Yukon either.)
> - Ranking: both 2000 and Yukon do not allow for merging of rankings
> from several tables/queries. However, it is important to us to be able
> to display results from several tables and sorted in a logical way. Any
> suggestions/work arounds?
> - Performance and scalability: How many rows and/or how many GB can I
> have in my DB and still get good FT search query performance (less than
> 5 seconds)? (I need data for both 2000 and Yukon)
> - Same with regards to indexing: ideally we would like to keep the
> index up to date in real time. We would like to use the track changes
> feature. However, our app allows many users to be logged in and
> edit/delete/add entries in the DB. What is the maximum amount of DB
> transactions per minute (second?) that will still allow us to keep the
> index updated in real time? (I need data for both 2000 and Yukon)
> -
> I would greatly appreciate any suggestions/tips/info/workaround.
> Thanks!
>

Monday, March 12, 2012

Issue with SSIS Package Configurations in SqlServer

Hello everyone,

I am working on a SSIS project and I am facing an issue for getting the configuration settings of the package, once it is deployed and executed from SQL Server agent.

The package uses two configuration types: (listed bellow in the order they are appeared in the configuration editor)

Config1 - Xml configuration file - for storing the database connection string.

Config2 - SQL Server - for storing some user defined variables. It uses the same database as specified in Config1.

Everything works fine and the package uses the database configuration values as defined in Config2, if I execute it from Visual Studio,

However, the package doesn’t get the configuration settings from the database when I try to execute it as a SQL Agent job.

There aren’t any errors and the package executes all tasks successfully, using the connection object Config1 (the same we use to get the config parameters from the database) and the default values of the user defined variables.

It works ok, if I change Config2 to be of type XML configuration file.

There could be two problems:

1. SQL server agent doesn’t read the configuration from the database and I am not quite sure how to set this. In Agent/ Job step properties screen/ Configurations tab I can only browse for a config file. I can also use the command window and /CONFIGFILE option to specify xml file, but how to use it in a case of a database configuration? Is there a /CONFIGDATABSE option or /CONFIGFILE works with database connection as well. I tried with /CONFIGFILE and database connection, but it doesn’t seem to work.

2. SQL server agent doesn’t get the configurations in the specified order. In my case,

it could try to read Config2 first, but at that moment it doesn’t have the database connection from Config1 and it fails. Again, I am not sure how to set the sequence.

Thanks in advance for your comments.

ITHave you specified a full path for Config1?|||

Yes, I specified the full path for Config1. I used the configuration tab to select the file and in the command line window it shows the full path.

The package itself uses the connection from Config1 for the data flow tasks and they work without any issues.

|||

Hi,

What kind of user defined variables are you fetching from the database?

Can you try storing them in package level variables by using a Script Task?

Regards,

B@.ns

|||

I think you can discard option 2. The package configurations should be processed in the order in which they are stored.

It sounds like a permissions problem with the SQl Server Agent account.

Have you tried profiling (SQL Profiler) the package execution to see if it is generating any SQL errors trying to read the configuration table? If the package fails to read the configuration, it only raises a warning, not an error, so it is not always obvious when a configuration fails to load.

|||

I couldn’t find any errors or warnings during the package execution.

Moreover the same connection is used for all data flow tasks inside the package and the SSIS logging (it uses SSIS log provider for SQL) and all of them work without any issues.

As a workaround I can add a SQL script to read those variables from the database, but this will duplicate the configuration functionality.

The idea was to use the existing SSIS database configuration feature instead of building a similar one from scratch.

|||Are you changig the path of the Conf1 in Agent, different than one you have when building the package?|||I.T.

See if you this blog spot can help you identifying the issue:

http://rafael-salas.blogspot.com/2007/01/ssis-package-configurations-using-sql.html

It uses an environment variable instead of a file|||

Hi Rafael,

I checked your blog (actually this was one of the first materials I found when I started exploring the issue a week ago), but in our case we have some system restrictions for using environment variables.

In your example you mentioned also that you used it successfully with XML file.

I would like to ask you if you were to deploy it and execute it as a SQL agent job?

Thanks,

IT

|||Yes, I did, and as far as I remember it worked ok. The only problems I can remember are stupid things like the XML file was not accessible for the account running the SS Agent; not updating the connection string properly withing the XML file (hence trying to connect to the wrong instance), etc. But most of the times, the package logging will tell you if something is wrong with any of the configurations and even shows the order in which the configurations were applied.|||

Rafael,

I would like to ask you if you remember how you set the sql configuration in the SQL Agent job. Did you do something special for this?

I couldn’t see any available options to set explicitly the configuration as SQL server (the UI allows only to browse and select config file).

The package logging is set to log all events and it doesn’t show any errors during the execution.

You mentioned also that the package logging can tell the order of the configuration, but I couldn’t see that kind of information. Is there a special custom event for this?

Thanks,

IT

|||

I just ran some tests using XML and Table configurations (in that order) and works fine in BIDS and as an agent job.

I didn't have to do any special thing in the job step (CmdExec); just to a command like:

DTEXEC /FILE "{path}\Configurations demo 2.dtsx" /MAXCONCURRENT " -1 " /CHECKPOINTING OFF /REPORTING EWCDI

But what I could notice is that the messages about package configurations are not shown in te progress tab; I recently updated to SP2, and I wonder if that has been changed. I dind't even received any warning when I deliberated removed the xml config file...not even in the package logging|||

I. T. wrote:

I couldn’t find any errors or warnings during the package execution.

This is normal behaviour in my experience. I didn't get any warnings or errors either.

Issue with SSIS Package Configurations in SqlServer

Hello everyone,

I am working on a SSIS project and I am facing an issue for getting the configuration settings of the package, once it is deployed and executed from SQL Server agent.

The package uses two configuration types: (listed bellow in the order they are appeared in the configuration editor)

Config1 - Xml configuration file - for storing the database connection string.

Config2 - SQL Server - for storing some user defined variables. It uses the same database as specified in Config1.

Everything works fine and the package uses the database configuration values as defined in Config2, if I execute it from Visual Studio,

However, the package doesn’t get the configuration settings from the database when I try to execute it as a SQL Agent job.

There aren’t any errors and the package executes all tasks successfully, using the connection object Config1 (the same we use to get the config parameters from the database) and the default values of the user defined variables.

It works ok, if I change Config2 to be of type XML configuration file.

There could be two problems:

1. SQL server agent doesn’t read the configuration from the database and I am not quite sure how to set this. In Agent/ Job step properties screen/ Configurations tab I can only browse for a config file. I can also use the command window and /CONFIGFILE option to specify xml file, but how to use it in a case of a database configuration? Is there a /CONFIGDATABSE option or /CONFIGFILE works with database connection as well. I tried with /CONFIGFILE and database connection, but it doesn’t seem to work.

2. SQL server agent doesn’t get the configurations in the specified order. In my case,

it could try to read Config2 first, but at that moment it doesn’t have the database connection from Config1 and it fails. Again, I am not sure how to set the sequence.

Thanks in advance for your comments.

ITHave you specified a full path for Config1?|||

Yes, I specified the full path for Config1. I used the configuration tab to select the file and in the command line window it shows the full path.

The package itself uses the connection from Config1 for the data flow tasks and they work without any issues.

|||

Hi,

What kind of user defined variables are you fetching from the database?

Can you try storing them in package level variables by using a Script Task?

Regards,

B@.ns

|||

I think you can discard option 2. The package configurations should be processed in the order in which they are stored.

It sounds like a permissions problem with the SQl Server Agent account.

Have you tried profiling (SQL Profiler) the package execution to see if it is generating any SQL errors trying to read the configuration table? If the package fails to read the configuration, it only raises a warning, not an error, so it is not always obvious when a configuration fails to load.

|||

I couldn’t find any errors or warnings during the package execution.

Moreover the same connection is used for all data flow tasks inside the package and the SSIS logging (it uses SSIS log provider for SQL) and all of them work without any issues.

As a workaround I can add a SQL script to read those variables from the database, but this will duplicate the configuration functionality.

The idea was to use the existing SSIS database configuration feature instead of building a similar one from scratch.

|||Are you changig the path of the Conf1 in Agent, different than one you have when building the package?|||I.T.

See if you this blog spot can help you identifying the issue:

http://rafael-salas.blogspot.com/2007/01/ssis-package-configurations-using-sql.html

It uses an environment variable instead of a file|||

Hi Rafael,

I checked your blog (actually this was one of the first materials I found when I started exploring the issue a week ago), but in our case we have some system restrictions for using environment variables.

In your example you mentioned also that you used it successfully with XML file.

I would like to ask you if you were to deploy it and execute it as a SQL agent job?

Thanks,

IT

|||Yes, I did, and as far as I remember it worked ok. The only problems I can remember are stupid things like the XML file was not accessible for the account running the SS Agent; not updating the connection string properly withing the XML file (hence trying to connect to the wrong instance), etc. But most of the times, the package logging will tell you if something is wrong with any of the configurations and even shows the order in which the configurations were applied.|||

Rafael,

I would like to ask you if you remember how you set the sql configuration in the SQL Agent job. Did you do something special for this?

I couldn’t see any available options to set explicitly the configuration as SQL server (the UI allows only to browse and select config file).

The package logging is set to log all events and it doesn’t show any errors during the execution.

You mentioned also that the package logging can tell the order of the configuration, but I couldn’t see that kind of information. Is there a special custom event for this?

Thanks,

IT

|||

I just ran some tests using XML and Table configurations (in that order) and works fine in BIDS and as an agent job.

I didn't have to do any special thing in the job step (CmdExec); just to a command like:

DTEXEC /FILE "{path}\Configurations demo 2.dtsx" /MAXCONCURRENT " -1 " /CHECKPOINTING OFF /REPORTING EWCDI

But what I could notice is that the messages about package configurations are not shown in te progress tab; I recently updated to SP2, and I wonder if that has been changed. I dind't even received any warning when I deliberated removed the xml config file...not even in the package logging|||

I. T. wrote:

I couldn’t find any errors or warnings during the package execution.

This is normal behaviour in my experience. I didn't get any warnings or errors either.

Issue with SSIS Package Configurations in SqlServer

Hello everyone,

I am working on a SSIS project and I am facing an issue for getting the configuration settings of the package, once it is deployed and executed from SQL Server agent.

The package uses two configuration types: (listed bellow in the order they are appeared in the configuration editor)

Config1 - Xml configuration file - for storing the database connection string.

Config2 - SQL Server - for storing some user defined variables. It uses the same database as specified in Config1.

Everything works fine and the package uses the database configuration values as defined in Config2, if I execute it from Visual Studio,

However, the package doesn’t get the configuration settings from the database when I try to execute it as a SQL Agent job.

There aren’t any errors and the package executes all tasks successfully, using the connection object Config1 (the same we use to get the config parameters from the database) and the default values of the user defined variables.

It works ok, if I change Config2 to be of type XML configuration file.

There could be two problems:

1. SQL server agent doesn’t read the configuration from the database and I am not quite sure how to set this. In Agent/ Job step properties screen/ Configurations tab I can only browse for a config file. I can also use the command window and /CONFIGFILE option to specify xml file, but how to use it in a case of a database configuration? Is there a /CONFIGDATABSE option or /CONFIGFILE works with database connection as well. I tried with /CONFIGFILE and database connection, but it doesn’t seem to work.

2. SQL server agent doesn’t get the configurations in the specified order. In my case,

it could try to read Config2 first, but at that moment it doesn’t have the database connection from Config1 and it fails. Again, I am not sure how to set the sequence.

Thanks in advance for your comments.

ITHave you specified a full path for Config1?|||

Yes, I specified the full path for Config1. I used the configuration tab to select the file and in the command line window it shows the full path.

The package itself uses the connection from Config1 for the data flow tasks and they work without any issues.

|||

Hi,

What kind of user defined variables are you fetching from the database?

Can you try storing them in package level variables by using a Script Task?

Regards,

B@.ns

|||

I think you can discard option 2. The package configurations should be processed in the order in which they are stored.

It sounds like a permissions problem with the SQl Server Agent account.

Have you tried profiling (SQL Profiler) the package execution to see if it is generating any SQL errors trying to read the configuration table? If the package fails to read the configuration, it only raises a warning, not an error, so it is not always obvious when a configuration fails to load.

|||

I couldn’t find any errors or warnings during the package execution.

Moreover the same connection is used for all data flow tasks inside the package and the SSIS logging (it uses SSIS log provider for SQL) and all of them work without any issues.

As a workaround I can add a SQL script to read those variables from the database, but this will duplicate the configuration functionality.

The idea was to use the existing SSIS database configuration feature instead of building a similar one from scratch.

|||Are you changig the path of the Conf1 in Agent, different than one you have when building the package?|||I.T.

See if you this blog spot can help you identifying the issue:

http://rafael-salas.blogspot.com/2007/01/ssis-package-configurations-using-sql.html

It uses an environment variable instead of a file|||

Hi Rafael,

I checked your blog (actually this was one of the first materials I found when I started exploring the issue a week ago), but in our case we have some system restrictions for using environment variables.

In your example you mentioned also that you used it successfully with XML file.

I would like to ask you if you were to deploy it and execute it as a SQL agent job?

Thanks,

IT

|||Yes, I did, and as far as I remember it worked ok. The only problems I can remember are stupid things like the XML file was not accessible for the account running the SS Agent; not updating the connection string properly withing the XML file (hence trying to connect to the wrong instance), etc. But most of the times, the package logging will tell you if something is wrong with any of the configurations and even shows the order in which the configurations were applied.|||

Rafael,

I would like to ask you if you remember how you set the sql configuration in the SQL Agent job. Did you do something special for this?

I couldn’t see any available options to set explicitly the configuration as SQL server (the UI allows only to browse and select config file).

The package logging is set to log all events and it doesn’t show any errors during the execution.

You mentioned also that the package logging can tell the order of the configuration, but I couldn’t see that kind of information. Is there a special custom event for this?

Thanks,

IT

|||

I just ran some tests using XML and Table configurations (in that order) and works fine in BIDS and as an agent job.

I didn't have to do any special thing in the job step (CmdExec); just to a command like:

DTEXEC /FILE "{path}\Configurations demo 2.dtsx" /MAXCONCURRENT " -1 " /CHECKPOINTING OFF /REPORTING EWCDI

But what I could notice is that the messages about package configurations are not shown in te progress tab; I recently updated to SP2, and I wonder if that has been changed. I dind't even received any warning when I deliberated removed the xml config file...not even in the package logging|||

I. T. wrote:

I couldn’t find any errors or warnings during the package execution.

This is normal behaviour in my experience. I didn't get any warnings or errors either.

Issue with SSIS Package Configurations in SqlServer

Hello everyone,

I am working on a SSIS project and I am facing an issue for getting the configuration settings of the package, once it is deployed and executed from SQL Server agent.

The package uses two configuration types: (listed bellow in the order they are appeared in the configuration editor)

Config1 - Xml configuration file - for storing the database connection string.

Config2 - SQL Server - for storing some user defined variables. It uses the same database as specified in Config1.

Everything works fine and the package uses the database configuration values as defined in Config2, if I execute it from Visual Studio,

However, the package doesn’t get the configuration settings from the database when I try to execute it as a SQL Agent job.

There aren’t any errors and the package executes all tasks successfully, using the connection object Config1 (the same we use to get the config parameters from the database) and the default values of the user defined variables.

It works ok, if I change Config2 to be of type XML configuration file.

There could be two problems:

1. SQL server agent doesn’t read the configuration from the database and I am not quite sure how to set this. In Agent/ Job step properties screen/ Configurations tab I can only browse for a config file. I can also use the command window and /CONFIGFILE option to specify xml file, but how to use it in a case of a database configuration? Is there a /CONFIGDATABSE option or /CONFIGFILE works with database connection as well. I tried with /CONFIGFILE and database connection, but it doesn’t seem to work.

2. SQL server agent doesn’t get the configurations in the specified order. In my case,

it could try to read Config2 first, but at that moment it doesn’t have the database connection from Config1 and it fails. Again, I am not sure how to set the sequence.

Thanks in advance for your comments.

ITHave you specified a full path for Config1?|||

Yes, I specified the full path for Config1. I used the configuration tab to select the file and in the command line window it shows the full path.

The package itself uses the connection from Config1 for the data flow tasks and they work without any issues.

|||

Hi,

What kind of user defined variables are you fetching from the database?

Can you try storing them in package level variables by using a Script Task?

Regards,

B@.ns

|||

I think you can discard option 2. The package configurations should be processed in the order in which they are stored.

It sounds like a permissions problem with the SQl Server Agent account.

Have you tried profiling (SQL Profiler) the package execution to see if it is generating any SQL errors trying to read the configuration table? If the package fails to read the configuration, it only raises a warning, not an error, so it is not always obvious when a configuration fails to load.

|||

I couldn’t find any errors or warnings during the package execution.

Moreover the same connection is used for all data flow tasks inside the package and the SSIS logging (it uses SSIS log provider for SQL) and all of them work without any issues.

As a workaround I can add a SQL script to read those variables from the database, but this will duplicate the configuration functionality.

The idea was to use the existing SSIS database configuration feature instead of building a similar one from scratch.

|||Are you changig the path of the Conf1 in Agent, different than one you have when building the package?|||I.T.

See if you this blog spot can help you identifying the issue:

http://rafael-salas.blogspot.com/2007/01/ssis-package-configurations-using-sql.html

It uses an environment variable instead of a file|||

Hi Rafael,

I checked your blog (actually this was one of the first materials I found when I started exploring the issue a week ago), but in our case we have some system restrictions for using environment variables.

In your example you mentioned also that you used it successfully with XML file.

I would like to ask you if you were to deploy it and execute it as a SQL agent job?

Thanks,

IT

|||Yes, I did, and as far as I remember it worked ok. The only problems I can remember are stupid things like the XML file was not accessible for the account running the SS Agent; not updating the connection string properly withing the XML file (hence trying to connect to the wrong instance), etc. But most of the times, the package logging will tell you if something is wrong with any of the configurations and even shows the order in which the configurations were applied.|||

Rafael,

I would like to ask you if you remember how you set the sql configuration in the SQL Agent job. Did you do something special for this?

I couldn’t see any available options to set explicitly the configuration as SQL server (the UI allows only to browse and select config file).

The package logging is set to log all events and it doesn’t show any errors during the execution.

You mentioned also that the package logging can tell the order of the configuration, but I couldn’t see that kind of information. Is there a special custom event for this?

Thanks,

IT

|||

I just ran some tests using XML and Table configurations (in that order) and works fine in BIDS and as an agent job.

I didn't have to do any special thing in the job step (CmdExec); just to a command like:

DTEXEC /FILE "{path}\Configurations demo 2.dtsx" /MAXCONCURRENT " -1 " /CHECKPOINTING OFF /REPORTING EWCDI

But what I could notice is that the messages about package configurations are not shown in te progress tab; I recently updated to SP2, and I wonder if that has been changed. I dind't even received any warning when I deliberated removed the xml config file...not even in the package logging|||

I. T. wrote:

I couldn’t find any errors or warnings during the package execution.

This is normal behaviour in my experience. I didn't get any warnings or errors either.

Wednesday, March 7, 2012

issue with a login name

Hi All,
Our network admin logged on to a machine with a Windows
Advanced Server OS as an administrator and installed SQL
Server 2000.
In the dialog box for the Account Services he selected
the system account option (not the Local). He entered my
login name and my password for the user login and the
password (mitra/mitra)
I logged on to the machine with an administrator login
and password (not my user login and password that we
entered for the services). I created a database using
MSSQL Enterprise Manager. I clicked "Login" in
the "Security" folder, there I saw three login names:
<Builtin\administrator>
<domain_name\mitra>
<sa>
I created a database. I clicked "Users" in my new
database. I did not see the <domain_name\mitra> being
listed as a user. Then I right-clicked on the "User" and
added this <domain_name\mitra> as a user to my new
database (the login name was already listed in the Login
drop-list).
I launched our application and tried to create the
database schema using <domain_name\mitra> for the login
name. I kept getting a sql message that the login is
invalid. I tried the <sa> login, no problem, i was able
to create my database schema.
Can someone please tell me why i could not create the
database schema using <domain_name\mitra> login name.
Thanks,
--Mitra> I launched our application and tried to create the
> database schema using <domain_name\mitra> for the login
> name. I kept getting a sql message that the login is
> invalid. I tried the <sa> login, no problem, i was able
> to create my database schema.
> Can someone please tell me why i could not create the
> database schema using <domain_name\mitra> login name.
I would expect a different error message (e.g. 'CREATE TABLE permission
denied'), given the steps you described. This is because a user needs to be
granted CREATE permissions in order to create a database object. If the
objects are to be owned by that user, you can grant the user CREATE
permission for the desired object types. For example:
GRANT CREATE TABLE TO [domain_name\mitra]
In order to create objects in a different schema (e.g. 'dbo'), the user
needs to be a member of one of the roles below. These roles have
permissions to create objects in any schema:
1) sysadmin server role
2) db_owner fixed database role
3) db_ddladmin fixed database role
Hope this helps.
Dan Guzman
SQL Server MVP
"mitra fatolahi" <mitra928@.hotmail.com> wrote in message
news:11af101c410a3$d08e31f0$a301280a@.phx
.gbl...
> Hi All,
> Our network admin logged on to a machine with a Windows
> Advanced Server OS as an administrator and installed SQL
> Server 2000.
> In the dialog box for the Account Services he selected
> the system account option (not the Local). He entered my
> login name and my password for the user login and the
> password (mitra/mitra)
> I logged on to the machine with an administrator login
> and password (not my user login and password that we
> entered for the services). I created a database using
> MSSQL Enterprise Manager. I clicked "Login" in
> the "Security" folder, there I saw three login names:
> <Builtin\administrator>
> <domain_name\mitra>
> <sa>
> I created a database. I clicked "Users" in my new
> database. I did not see the <domain_name\mitra> being
> listed as a user. Then I right-clicked on the "User" and
> added this <domain_name\mitra> as a user to my new
> database (the login name was already listed in the Login
> drop-list).
> I launched our application and tried to create the
> database schema using <domain_name\mitra> for the login
> name. I kept getting a sql message that the login is
> invalid. I tried the <sa> login, no problem, i was able
> to create my database schema.
> Can someone please tell me why i could not create the
> database schema using <domain_name\mitra> login name.
> Thanks,
> --Mitra
>

Monday, February 20, 2012

Issue doing Integrated services programming using Microsoft.SqlServer.ManagedDTS.dll

Hi,

I am writing an installer code in C# to deploy the SSIS package.

I want to use Microsoft.SqlServer.ManagedDTS.dll for it.

It is mentioned in few articles available online that Microsoft.SqlServer.ManagedDTS.dll ships with SQL Server 2005.

I searched on our database server but could not get it.

Anyone having idea on this please help.

HV

Don't try and install SSIS by hand, it is not a good idea It is not supposed to be a redistributable component. There are a lot more assemblies that just that one for SSIS.

Since SSIS requires a full SQL Server license, why not use the regular SQL setup to do this for you. You can choose which servers you want.

If you have installed SSIS on your server then it will certainly be in the GAC, but it may not be on the file system outside of this, unless you installed the tools, in which case it is - C:\Program Files\Microsoft SQL Server\90\SDK\Assemblies

|||Another question to ask: Is SSIS installed on this SQL Server box? A lot of SQL 2005 servers will be set up by their DBAs to not have any unnecessary components installed, and SSIS sometimes falls into this category.|||

Hi,

Thank you Darren.

SSIS is installed on the machine.

I checked the following folder :- C:\Program Files\Microsoft SQL Server\90\SDK\Assemblies

but dll is not available.

Which tool installation delivers Microsoft.SqlServer.ManagedDts.dll?

Please let me know.

HV

|||

Hi,

Thank you Mathew.

SSIS is installed on the machine.

I checked the following folder :- C:\Program Files\Microsoft SQL Server\90\SDK\Assemblies

but it's not available.

HV

|||

Hi,

I picked the Microsoft.SQLServer.ManagedDTS.dll from following folder:

C:\WINDOWS\assembly\GAC_MSIL\Microsoft.SqlServer.ManagedDTS\9.0.242.0__89845dcd8080cc91>

Similarly picked Microsoft.SqlServer.DTSRuntimeWrap.dll also.

I added it as reference in my .NET application.

When I execute the program I get below error:

Retrieving the COM class factory for component with CLSID {E44847F1-FD8C-4251-B5DA-B04BB22E236E} failed due to the following error: 80040154.

Any Clue?

How to get the RunningPackages information back to a client PC?

HV

|||

If you want running packages information, why not just use the MS tools?

Do you have SSIS tools installed on the local machine that hosts the program? The error indicates that Microsoft.SqlServer.DTSRuntimeWrap is not installed on the local PC?

|||

No SSIS is not installed on the machine which hosts program.

But that is my requirement , I want to deploy SSIS package remotely.

I have installed Microsoft.SQLServer.DTSRuntimeWrap.dll in GAC.

Let me know if there is a way out to use it on a machine where SSIS is not installed.

HV

|||

SSIS is not installed you say, and you get an error that says in cannot find a COM component. Do you think there may be a connection?

I refer you to my original post, apart from saying that a manual install was a silly idea, I also pointed out "There are a lot more assemblies that just that one for SSIS."

As a start point that DLL is just a wrapper to the COM library that does the work, the name hints at that, and the error proves that it is trying to use a COM DLL that is not there, a COM DLL with a ProgID of {E44847F1-FD8C-4251-B5DA-B04BB22E236E} perhaps. As I said before there are lots of DLLs involved in SSIS not just one, so use a proper install.

What you are asking for a not a supported scenario, you may be violating your license agreements if not careful, and at any rate will be a very complicated task to try and reverse engineer the requirements, and very slow if you cannot track a missing COM DLL down yourself.

If you think this is wrong, post feedback to MS (http://connect.microsoft.com) telling them why you think you should have some redistribuatable support, but in the mean-time you will need to run one of the MS installs to get this to work.

Why do you not want to use a MS install?