I have blogged previously at SSDT – What’s in a name? about the naming confusion that Microsoft have created by their inconsistent use of the term “SQL Server Data Tools” (SSDT) in SQL Server 2012. Is it a Visual Studio shell incorporating project templates for building databases, SSIS packages, SSAS cubes & SSRS reports? Or is it simply the project template for databases? Pick any two people on the SQL Server team and they’ll probably give you two different (and probably ambiguous) answers.

The situation infuriates the hell out of me but probably not as much as it does this guy who posted to the SSDT forum:

I installed SSDT 2012 (December version) on an existing Visual Studio 2012 Premium install but I can't find the BI templates!!!?

When informed that the BI project templates (i.e. SSIS, SSAS, SSRS) only come with a SQL Server installation and that actually they’re (still) not available for VS2012 he had this to say:

I wish this was better documented so that I could have saved a few hours.

That’s a guy who has wasted hours of his life simply because Microsoft can’t name their products properly and sensibly. Is it really that difficult?

Rant over. Sorry, but it needed to be said.

@Jamiet

UPDATE. A day after publishing this blog post and I stumble across someone asking a question about SSIS on the SSDT forum. The SSDT forum is for questions about SSDT database projects but of course, this person has read all the documentation from Microsoft saying that SSIS is part of SSDT and hence posts his SSIS questions on the SSDT forum. This is bad. Bad bad bad.

Continuous deployment is described by Wikipedia as:

Most CI systems allow the running of scripts after a build finishes. In most situations, it is possible to write a script to deploy the application to a live test server that everyone can look at. A further advance in this way of thinking is Continuous Deployment, which calls for the software to be deployed directly into production
http://en.wikipedia.org/wiki/Continuous_Integration

I think of Continuous Deployment as a natural extension of Continuous Integration where not only do we build the source code, we deploy it as well. As I continue to put together my SSDT Database Projects from the ground-up seminar it occurred to me that demonstrating Continuous Deployment for SSDT database projects would be very useful. It then occurred to me that the recently released Team Foundation Service (TFS) includes a facility to build source code so I wondered, could one perhaps use TFS to build an SSDT database project and deploy it to a Windows Azure SQL Database (aka SQL Azure)? It turns out that the answer is “yes” and this blog provides a step-by-step guide to doing just that.

Its worth noting that Team Foundation Service is free for up to five users. SSDT is also free. Windows Azure SQL Databases, however, are not free so you may have to pay a small amount to get one or alternatively adapt the steps herein to use Team Foundation Server to deploy to an on-premises SQL Server instance. Note that a MSDN subscription does include some small usage of Windows Azure SQL Database and that allowance will be more than enough to go through the steps herein.

We can break our process down into the following high-level steps:

• Create a Windows Azure SQL Database
• Sign up for Team Foundation Service (TFS)
• Create an SSDT database project
• Add the SSDT project to TFS Source Control
• Setup your Publish Profile file
• Create a Build Definition

Let’s dive into all of those steps.

Create a new Windows Azure SQL Database

Visit https://manage.windowsazure.com/ and sign up for a new Windows Azure SQL Database:

Your new database will be viewable in the portal immediately:

Windows Azure SQL Databases are, by default, closed to the outside world so you must edit the list of IP addresses that are allowed to access the server. Setting this up is outside the scope of this blog post however simply go to How to: Configure the Server-Level Firewall Settings (Windows Azure SQL Database) to learn how to do this (its very easy).

Sign up for Team Foundation Service

Visit https://tfs.visualstudio.com/ to sign up to use Team Foundation Service using your Microsoft Account (aka Windows Live ID). Once you have signed-up you will have a dedicated service URL (mine, for example,  is https://jamiet.visualstudio.com/). Visit that URL and click “New Team Project”:

Fill in the pertinent details and hit “Create Project”:

Creating the Team Project will take a minute or so but will be ready for you when you need it shortly.

Create an SSDT database project

For this you’re going to need SSDT for Visual Studio 2010 or Visual Studio 2012. They are both free and can be downloaded from here; I am using SSDT for Visual Studio 2012.

You will need to create a new SQL Server Database Project:

Once the project is created it will appear in Solution Explorer and be empty at this point. We are going to be deploying to Windows Azure SQL Database therefore we must tell the project that that will be the case. To do that right-click on the project and select Properties:

In Project Settings select Windows Azure SQL Database as the Target platform:

You’ll need to add a database object to your project otherwise this is all rather pointless. Given that this is simply for demo purposes we shall simply create a table called [Table1].

OK, we now have a Windows Azure SQL Database, a TFS service (is that Team Foundation Service service???) and a SSDT database project with a simple table in it. Let’s now hook them all together.

Add the SSDT project to TFS Source Control

Interaction with TFS from within Visual Studio is done using the Team Explorer pane.

From Team Explorer you will need to connect to your new TFS. Click on the address bar in Team Explorer and point to “Connect to Team Projects…”

Choose your project and click Connect. Your server is simply the name of the server that was setup earlier; remember that mine was jamiet.visualstudio.com (note that it uses an https connection, not http):

You should now be connected to your Team Foundation Service. Add your source code to TFS Source Control from inside Solution Explorer:

And check in your SSDT database project (or “commit” if you prefer that terminology):

You will now be able to browse to your TFS home page (mine is https://jamiet.visualstudio.com), click on your project and browse through your checked in code:

Setup your Publish Profile file

You need to tell SSDT where to deploy the project to, that information is stored in a Publish Profile file. In Solution Explorer right-click and select “Publish…”

Enter the details of the Windows Azure SQL Database that you prepared earlier:

Once the Publish Profile file has been saved it will be saved in your project:

At the time of writing Windows Azure SQL Database only supports SQL authentication so we have to edit the Publish Profile file to include the password for the account that will be used for deployment. Right-click on the Publish Profile file, select “Open with…” and from the resultant dialog select “XML (Text) Editor:

Edit the Connection String to include your password:

NOW you can check in:

Create a Build Definition

A Build Definition defines how and when the source code in the project should get built. Create a new Build Definition by selecting “Builds” in the Team Explorer address bar and choosing New Build Definition:

Follow the steps shown in the screenshots below to setup your Build Definition:

(Credit for informing me about the MSBuild arguments goes entirely to Jakob Ehn and his blog post Deploying SSDT Projects with TFS Build)

And you’re done!

If you have set everything up correctly then any future check in should trigger a build and thus a Publish of the SSDT project to your Windows Azure SQL Database. Here is my first build report:

and back in SSDT I can use SQL Server Object Explorer to browse my newly deployed table:

That’s Continuous Deployment for SSDT projects to Windows Azure SQL Database using Team Foundation Service. Awesome!

@Jamiet

A couple of weeks ago in Get to Know SQL Server 2012's SQL Server Data Tools [article published] I stated that an article that I wrote about SSDT had been published and was available for reading online at http://www.devproconnections.com/. The article has since been updated to include a 10-minute video that I recorded demonstrating some of the basic concepts of SSDT so if you want to hear me drone on about SSDT in my dulcet Yorkshire brogue (!!!), click through, settle back and enjoy!

@Jamiet

SQL Server Data Tools (SSDT) is released on a very regular cadence (note that I’m talking about the database projects part of SSDT, this is nothing to do with SSIS/SSAS/SSRS). Since the first release in March 2012 there have been releases in September 2012, November 2012, & December 2012. I personally have found it difficult to keep track of not only which particular version I am using but also that which are my colleagues are using – quite simply I didn’t know how to discover the current version number. I assumed that the answer would lie within Visual Studio 2012 in the Help->About Microsoft Visual Studio menu option, when I checked there I saw this:

A version number – 11.1.21208.0. What does that mean though? To understand more we have to go to Add/Remove programs:

There are many different items listed there, let’s look at them in turn:

• Microsoft SQL Server 2012 Data-Tier App Framework. This is fairly self-explanatory if you know what the Data-Tier App Framework (aka DACFx) is. If you don’t, see the notes at the end of this blog post*. The reason that you see two entries in the list above is due to my having the 32bit & 64bit versions installed – the fact that there is no differentiation between the two in the title is a bug.
• Microsoft SQL Server 2012 Express LocalDB. LocalDB is the small server-less version of SQL Server that now gets deployed with SQL Server proper. LocalDB also gets installed with SSDT if you do not already have it, but you should consider it part of SQL Server itself.
• Microsoft SQL Server Data Tools 2010. This was described to me as the “installation chainer” – it is the installation component that manages SSDT and its dependencies. I think of it as the SSDT shell within Visual Studio 2010.
• Microsoft SQL Server Data Tools 2012. This was described to me as the “installation chainer” – it is the installation component that manages SSDT and its dependencies. I think of it as the SSDT shell within Visual Studio 2012. Note that the version number is the same as in the Help->About screenshot at the top of this article – that is because they are one and the same.
• Microsoft SQL Server Data Tools – Database Projects – Web installer entry point. A stub project that gets installed when you install SQL Server. Its the SSDT database project entry point.
• Microsoft SQL Server Data Tools – enu (10.3.21208.0)& Microsoft SQL Server Data Tools – enu (11.1.21208.0). These are the things we’re interested in. These *are* SSDT database projects, for Visual Studio 2010 & Visual Studio 2012 respectively. The version numbers are those for the December 2012 release; note that the build numbers are the same (X.X.21208.X), only the major and minor numbers are different.
• Microsoft SQL Server Data Tools Build Utilities – enu (10.3.21208.0)& Microsoft SQL Server Data Tools Build Utilities – enu (11.1.21208.0). Everything you need to do a headless (i.e. without having Visual Studio installed) build/deploy of an SSDT database project.

So at the end of all that we have determined the following version numbers for SSDT database projects are:

I do not know the version numbers for the March 2012, September 2012 & November 2012 releases (if anyone out there does know please let me know in the comments) however I shall update this blog post for future releases. September 2012 and November 2012 releases now added!

I have requested that any blog post to announce future releases should contain the version numbers. Let’s hope that happens.

@Jamiet 

* DACFx is described here as:

The Microsoft® SQL Server® 2012 Data-Tier Application Framework (DACFx) is a component which provides application lifecycle services for database development and management for Microsoft® SQL Server® and Windows Azure SQL Databases.
DacFX supports various database deployment and management scenarios for SQL Server and Windows Azure SQL Databases including extracting/exporting a live database to a DAC package, deploying a DAC package to a new or existing database, and migrating from on-premise SQL Server to Windows Azure. This functionality is exposed via the DACFx managed API. DACFx can target SQL Server 2005 SP4, 2008 SP1, 2008R2, 2012, and Windows Azure SQL Databases. DACFx also provides the command-line utility SqlPackage.exe for creating and deploying .dacpac and .bacpac packages. DACFx supports scenarios provided by SQL Server client tooling including SQL Server Data Tools and SQL Server Management Studio 2012.

Interesting the stuff you pick up from reading forums…

“The BATCH seperators [in SSDT deployment scripts] are used to provide scoping when resolving object shapes. SSDT or VSDB scripts for that matter serve a different purpose, they provide the blueprint of the model you want, and do not represent how it will be used to construct the target site. We choose TSQL as our DSL, at first we wanted to use a different representation, but user research in 2005 told us loud and clear users wanted to use TSQL as their DSL. In order to make the compiler work, we had to impose some simplifications, one is a more frequent use of BATCH seperators to indentify object scopes.”

by Gert Drapers (former lead of the SSDT product team) at http://social.msdn.microsoft.com/Forums/en-US/ssdt/thread/47d4124b-82cd-48f9-8ba7-c2ae4c73cbcf/

Personally I’m glad that they chose to use T-SQL as it makes debugging easier but I’d be interested to know what else was on the table!

@Jamiet

Yesterday I attended SQL Saturday 194 in Exeter for which many people deserve credit but especially the organisers Jonathan Allen& Annette Allen and the sponsors Fusion-IO, Confio Software, Nexus, Big Bang Data Company, Purple Frog Systems, Redgate, idera, Pluralsight, Jurys Inn Exeter& Wrox.

would you have any extra references on how to handle client-specific changes to the DB schema

I promised Leo an answer, hence this blog post.

If I understand Leo’s question correctly he has an application that is used by multiple clients, he has a separate instance of the application for every client, and the database requires client-specific customisations; luckily for Leo there is a new feature in SSDT that is perfect for this situation. First let’s set up our solution in SSDT:

We have three database projects:

• _core - This contains all the objects that are common to all clients. For demonstration purposes it simply contains a table called [Sales]
• clientFoo – All the objects that are required only by client “Foo”. In this case there is view which aggregates the data in [Sales] by Customer
• clientBar – All the objects that are required only by client “Bar”. In this case there is view which aggregates the data in [Sales] by Location

In order that the reference to table [Sales] in the two views can be resolved both of the client-specific projects have a database reference to _core:

It is those database references that allows objects in clientFoo/clientBar to refer to objects in _core. Now here’s the important bit. When we set up those two database references we must specify that the objects in the referenced project are intended to be in the same database:

Having objects from one database split over multiple projects is called composite projects. The effect of this is that (by default) whenever clientFoo or clientBar are deployed the objects in _core will get deployed as well. We can see this in the output when we deploy clientFoo:

Notice how even though we chose not to deploy _core two objects have been created; table [Sales] (from the _core project) and view [vSalesPerCustomer]. This is the new feature in SSDT that I spoke of above – a deployment of a project will (by default) also deploy all the objects in referenced projects where objects in the referenced database are intended to be in the same database (incidentally this functionality replaces the “partial projects” feature from previous incarnations of SSDT).

Note that you can change this default behaviour in the advanced publish settings by unchecking the “Include composite objects” box:

Hence, with the “Include Composite Objects” setting turned on, we can deploy both projects clientFoo & clientBar and both will contain table [Sales] plus their own client-specific view:

which (I hope) is exactly what Leo was after!

That’s all there is to it. A very very nice new feature of SSDT!

@Jamiet

When you install the database projects template of SSDT you get SQL Server Object Explorer (SSOX) installed as well. SSOX is a pane within Visual Studio and is the main enabler of the Connected Development experience that the SSDT team have attempted to provide.

SSOX provides some really cool capabilities that are not in SQL Server Management Studio (I hope to blog about them in the near future). In theory these capabilities make it possible for a database developer to spend all their time in SSDT (i.e. Visual Studio) thus making SSMS a pureplay DBA tool (this does of course depend on your definition of both a database developer and a DBA, but I’m not getting into that debate here).

With that in mind I have spent a few days trying to work without SSMS, preferring to live wholly inside Visual Studio instead. By and large I was able to do everything I needed to do from within Visual Studio however there were a few nuances about the experience that kept pushing me back to SSMS, I detail those nuances below.

Server groups

SSOX combines the functions of SSMS’s Object Explorer and Registered Servers pane. I don’t mind either way of working but it does mean that there is no ability to group servers in SSOX like you can in the Registered Servers pane

F6

In SSMS I regularly use the F6 keyboard shortcut to jump between the query, results & messages panes of a query window. No such keyboard shortcut exists in SSDT and they’ve already canned my request on Connect to get this fixed (even though it laughably has status “closed as fixed”).

Change Connection

The context menu in SSMS provides the ability to change a connection as well as connect and disconnect:

SSDT doesn’t have change connection and believe me, you don’t know how much you use a feature until its not there:

There’s also no hotkey to jump to “Connection” on the context menu like there is in SSMS (“C”) and I find that annoying too.

@Jamiet

As you may have realised from much of my blogging over the past year or so I’m an advocate of using SSDT database projects for building database solutions on SQL Server. I have been using SSDT database projects a lot in that time and have come up with a checklist of things to consider when starting a new SSDT database project and I’ll be detailing that checklist below. I strongly advise you to consider making decisions about these items before you even write a scrap of code as invariably it will be more difficult to change later, especially if you have already deployed your database.

In no particular order here is my checklist:

Folder Structure

By default SSDT will not provide a folder structure for new projects in which to store all your script files so you might want to consider setting one up yourself. I recommend not trying to create a large hierarchy of folders to start with as this will evolve as you go about building your database. Here’s a simple starter for ten:

Filegroups

If you don’t understand the importance of filegroups then read Thomas Larock’s recent blog post DATABASE FILEGROUPS: JUST LIKE SEATBELTS BUT WITH LESS CHAFING. Quite often your organisation will stipulate policies for what filegroups you should be using and I highly recommend that you set up filegroups in your database projects to match those policies sooner rather than later. If your organisation does not have any such policies then perhaps think about defining them yourself, Thomas lists a few considerations that may influence your decisions:

• separation of system data from user data
• larger indexes may benefit from their own filegroup
• archival of data can benefit from using dedicated filegroups as this will reduce backup maintenance tasks

If you follow my suggested folder structure above then put your filegroups into the “Storage” folder:

Above all, specify your default filegroup especially as many DBAs won’t be happy about you putting objects into the PRIMARY filegroup (which is the default). This is done by right-clicking on the project and selecting “Properties..” When there hit “Database Settings…” and set your default filegroup on the “Operational” tab:

If you intend to use the filestream feature of SQL Server then you can set the default filestream filegroup here too.

Collation

If you take only one piece of advice from this blog post make it this: set your collation before you write a single line of code (just trust me on this, OK). You’ll find this in Project Properties->Project Settings->Database Settings->Common

Again, find out if your organisation has any guidance on database collation.

Specify Target platform

SSDT allows you to deploy a project to SQL Server 2005, 2008, 2012 or Azure so it stands to reason that you should specify which you are intending to deploy to. You’ll find this in Project Properties->Project Settings:

Default Schema

If you follow the premise that “anything that SQL Server picks as the default is probably wrong” then you should probably think about changing the default schema which, by default, will be [dbo]. SSDT will create all new objects in the default schema (unless otherwise specified).

Generally I’m of the opinion that not accepting SQL Server’s defaults is a good idea if only because it forces you to think about these things and be aware of them; in the case of schemas it forces you to think about security and who should have GRANT or DENY permissions and on what.

Specify the default schema in Project properties->Project Settings:

Review other database settings

I’ve mentioned the most important database settings that you should look to change however you should also glance over the rest of the defaults that SSDT chooses for you to verify that they are valid for your project. Again, these are in Project Properties->Project Settings->Database Settings. a few I’d call out as being especially worthy of your attention are:

• Auto update stats (on by default)
• Recovery model (FULL by default)
• Transaction Isolation (RCSI off by default)
• Service broker (disabled by default)

Code Analysis

SSDT always checks the syntax of your database code, that’s one of the main justifications for using SSDT database projects. However what it does not do (not by default anyway) is check whether the code you write might be considered good code. That’s what Code Analysis is for, your code gets checked to see if it adheres to well-understood good practices for SQL Server database development. For example, Code Analysis will check for use of “SELECT*” which is generally regarded as a bad thing.

Code analysis is turned off by default. I recommend that you head into Project Properties->Code Analysis and turn it on. Moreover I recommend that you check all the boxes under “Treat Warning as Error” – this forces developers to address issues that get raised by Code Analysis.

Big Labels

Add a banner to your Pre-Deployment script as I describe in Big label generator. This may seem rather unnecessary but it takes about 30 seconds and believe me, at some point you’ll be glad you did!

The demo project that I took some of the above screenshots from is downloadable from http://sdrv.ms/ZW0gNt and might provide a useful template for your own SSDT database projects.

If you have any more suggestions for my checklist please add them to the comments below!

@Jamiet

In March 2012 I published SSDT - What's in a name? where I lamented the mistakes that Microsoft made in the naming of SQL Server Data Tools.

…official documentation stating that SSDT includes all the stuff for building SSIS/SSAS/SSRS solutions (this is confirmed in the installer, remember) yet someone from Microsoft tells him "SSDT doesn't include any BIDs components".

I have been close to this for a long time (all the way through the CTPs) so I can kind of understand where the confusion stems from. To my understanding SSDT was originally the name of the database dev stuff but eventually that got expanded to include all of the dev tools - I guess not everyone in Microsoft got the memo.

Since then I’ve seen lots of questions pertaining to SSIS/SSAS/SSRS being posted on the SSDT forum on MSDN which, frankly, is the wrong place for them.

With the release of the SSIS/SSAS/SSRS project templates for Visual Studio 2012 Microsoft have attempted to clear up the confusion. Matt Masson from the SSIS product team attempts to explain in his usual jovial way:

note that the component was renamed – we added “Business Intelligence” to the end to distinguish it from the SQL Server Data Tools (Juneau). We now refer to it as SSDTBI, rather than “SSDT, no, not that one, the other one – you know, the one that comes with the SQL installation media, not the one you download”.
http://www.mattmasson.com/2013/04/installing-ssis-for-visual-studio-2012/

So to clarify, its now:

• SSDT – for building databases
• SSDTBI – for building SSIS/SSAS/SSRS solutions

Got it? Good!

That may all seem slightly confusing but its a darn sight clearer than it was SQL Server 2012 was released over a year ago. And if nothing else you have to be amused with Microsoft’s penchant for ever-lengthening acronyms, I can’t recall ever seeing a six-letter-acronym before, can you?

@Jamiet

One of the nice things about doing user group presentations is that when you’re putting the presentation together you invariably learn about features that were previously unbeknown to you; so it proved as I stumbled upon SSDT’s database drift detection features while researching material for my forthcoming pre-conference seminar SSDT from the ground up.

What is database drift?

You have probably experienced database drift, you just didn’t happen to refer to it as that. More likely you might have spluttered the following, perhaps sprinkled with a few expletives:

• “Who put these tables in my database?”
• “Who changed this view definition?”
• “Why is this guy in db_owner?”
• “Where has my stored procedure gone?”

In other words database drift can loosely be described as

stuff that appears, gets removed, or gets modified in your production databases that perhaps shouldn’t be

Detecting database drift using SSDT

If you’re using SSDT to manage your database schema then you probably consider the source code in your SSDT projects to be “the truth” and hence anything that appears in your databases that is not in your source code would be considered database drift.

In order to detect database drift using SSDT you must ensure that your database is registered as a Data-Tier Application. This can be done when you publish your database project (i.e. dacpac) by selecting “Register as a Data-tier Application”:

Thereafter you can check for database drift on subsequent publishes by selecting “Block publish when database has drifted from registered version”:

If you check that box and database drift has occurred then the publish operation will fail and you see an appropriate message in the Data Tools Operations pane, “Publish stopped. The target database has drifted from the registered version.”:

Clicking the View Report hyperlink displays the Drift Report which is represented in an XML file:

<?xml version="1.0" encoding="utf-8"?><DriftReport xmlns="http://schemas.microsoft.com/sqlserver/dac/DriftReport/2012/02">  <Additions>    <Object Name="[View_1]" Parent="[dbo]" Type="SqlView" />    <ExtendedProperty HostName="[View_1]" HostParent="[dbo]" HostType="SqlView" Count="2" />  </Additions>  <Removals />  <Modifications /></DriftReport>

In this case a view called [dbo].[View_1] has been added to the target database. That view did not exist in the dacpac that was most recently deployed against the database thus the publish operation fails. Keeping one’s deployed databases as “clean” as possible is something that I am all in favour of so personally I think this is a pretty cool feature.

Generating a drift report from the command-line

The drift report can be generated by the command-line tool sqlpackage.exe. To do so you need to define:

• the action to be DriftReport
• a target server & database
• an output file

>SqlPackage.exe /A:DriftReport /tsn:"(localdb)\Projects" /tdn:"Database1" /op:DriftReport.xml

As far as I know there is no support for generating a drift report from SQL Server Management Studio (SSMS). I’m hoping that changes so that this feature gets more visibility.

If you have any comments stick them in the comments section below!

@Jamiet

SQL Server Data Tools (SSDT) has a neat feature where you can add snippets into your scripts via the right-click context menu:

I’m finding it very useful indeed. The same feature exists in SQL Server Management Studio (SSMS) as well by the way:

One thing I really wanted to be able to do was create my own snippets for SSDT and I ventured to the SSDT forum to ask if it was possible. Turns out the answer is “yes” and Gert Drapers replied to my thread by providing an excellent run through of how to do it. What Gert’s post didn’t quite clarify is that if you follow his instructions then you have to manually edit the .snippet file that you created so that the language is set to SQL_SSDT:

(well, I had to do that anyway)

Once you do that you’ll be able to import the snippet into Visual Studio (Gert’s post shows you how) and thereafter your snippets will show up in the snippets menu:

Very handy indeed.

@Jamiet

I have discovered what seems, to me, like a bug in the June 2013 release of SSDT and given the problems that it created yesterday on my current gig I thought it prudent to write this blog post to inform people of it.

I’ve built a very simple SSDT project to reproduce the problem that has just two tables, [Table1] and [Table2], and also a procedure [Procedure1]:

The two tables have exactly the same definition, both a have a single column called [Id] of type integer.

CREATE TABLE [dbo].[Table1]
(
    [Id] INT NOT NULL PRIMARY KEY
)

My stored procedure simply joins the two together, orders them by the column used in the join predicate, and returns the results:

CREATE PROCEDURE [dbo].[Procedure1]
AS
    SELECT t1.*
    FROM    Table1 t1
    INNER JOIN Table2 t2
        ON    t1.Id = t2.Id
    ORDER BY Id

Now if I create those three objects manually and then execute the stored procedure, it works fine:

So we know that the code works. Unfortunately, SSDT thinks that there is an error here:

The text of that error is:

Procedure: [dbo].[Procedure1] contains an unresolved reference to an object. Either the object does not exist or the reference is ambiguous because it could refer to any of the following objects: [dbo].[Table1].[Id] or [dbo].[Table2].[Id].

Its complaining that the [Id] field in the ORDER BY clause is ambiguous. Now you may well be thinking at this point “OK, just stick a table alias into the ORDER BY predicate and everything will be fine!” Well that’s true, but there’s a bigger problem here. One of the developers at my current client installed this drop of SSDT and all of a sudden all the builds started failing on his machine – he had errors left right and centre because, as it transpires, we have a fair bit of code that exhibits this scenario.  Worse, previous installations of SSDT do not flag this code as erroneous and therein lies the rub. We immediately had a mass panic where we had to run around the department to our developers (of which there are many) ensuring that none of them should upgrade their SSDT installation if they wanted to carry on being productive for the rest of the day.

Also bear in mind that as soon as a new drop of SSDT comes out then the previous version is instantly unavailable so rolling back is going to be impossible unless you have created an administrative install of SSDT for that previous version.

Just thought you should know! In the grand schema of things this isn’t a big deal as the bug can be worked around with a simple code modification but forewarned is forearmed so they say!

Last thing to say, if you want to know which version of SSDT you are running check my blog post Which version of SSDT Database Projects do I have installed?

@Jamiet

Microsoft released SQL Server Data Tools (SSDT) along with SQL Server 2012 in the Spring of 2012. Since then I’ve noticed an upward tick in both the number of organisations that are using SSDT and the number of questions that are getting asked about it on forums. There is some confusion about what SSDT actually is (Microsoft hasn’t helped there), why people should be using SSDT and, most importantly, how to make best use of it.

Its clear that people want to learn more about SSDT so I have joined forces with Technitrain to offer a 2–day training course, in London, in March 2014 called Introduction to SQL Server Data Tools

The course will cover:

If this sounds like your bag then please sign up on the Technitrain website.

@Jamiet

I discovered a little wrinkle in SSDT today that is worth knowing about if you are managing your database schemas using SSDT. In short, if a file is moved to a different folder in the project then any code analysis suppressions that reference that file will disappear from the suppression file. This makes sense if you think about it because the paths stored in the suppression file are no longer valid, but you probably won’t be aware of it until it happens to you. If you don’t know what code analysis is or you don’t know what the suppression file is then you can probably stop reading now, otherwise read on for a simple short demo.

Let’s create a new project and add a stored procedure to it called sp_dummy.

Naming stored procedures with a sp_ prefix is generally frowned upon and hence SSDT static code analysis will look for occurrences of this and flag them. So, the next thing we need to do is turn on static code analysis in the project properties:

A subsequent build causes a code analysis warning as we might expect:

Let’s suppose we actually don’t mind stored procedures with sp_ prefixes, we can just right-click on the message to suppress and get rid of it:

That causes a suppression file to get created in our project:

Notice that the suppression file contains a relative path to the file that has had the suppression placed upon it. Now if we simply move the file within our project to a new folder notice that the suppression that we just created gets removed from the suppression file:

As I alluded above this behaviour is intuitive because the path originally stored in the suppression file is no longer relevant but you’re probably not going to be aware of it until it happens to you and messages that you thought you had suppressed start appearing again. Definitely one to be aware of.

@Jamiet 

Last Thursday I presented my session “Introduction to SSDT” at the SQL Supper event held at the offices of 7 Digital (loved the samosas, guys). I did my usual spiel, tour of the IDE, connected development, declarative database development yadda yadda yadda… and at the end asked if there were any questions. One gentleman in attendance (sorry, can’t remember your name) raised his hand and stated that by attempting to evangelise all of the features I’d missed the single biggest benefit of SSDT, that it can tell you stuff about database that you didn’t already know.

I realised that he was dead right. SSDT allows you to import your whole database schema into a new project and it will instantly give you a list of errors and/or warnings pertaining to the objects in your database. Invalid references (e.g a long-forgotten stored procedure that refers to a non-existent column), unnecessary 3-part naming, incorrect case usage, syntax errors…it’ll tell you about all of ‘em! Turn on static code analysis (this article shows you how) and you’ll learn even more such as any stored procedures that begin with “sp_”, WHERE clauses that will kill performance, use of @@IDENTITY instead of SCOPE_IDENTITY(), use of deprecated syntax, implicit casts etc…. the list goes on and on.

I urge you to download and install SSDT (takes a few minutes, its free and you don’t need SQL Server or Visual Studio pre-installed), start a new project:

right-click on your new project and import from your database:

and see what happens:

You may be surprised what you discover. Let me know in the comments below what results you get, total number of objects, number of errors/warnings, I’d be interested to know!

@Jamiet

Yesterday on Twitter Ryan Desmond asked Is there a good read for #SSDT regarding deploying changes via schema compare vs solution deployment?

I don’t know of any article that covers this so in this blog post I offer my opinion on the subject.

First some background. When building databases offline using the database project type (.sqlproj) in SSDT you have two options for deploying the DDL code in your project into a physical database:

• Schema Compare
• Publishing

Under the covers both do the same basic operation; build a dacpac from your project, compare it to the target database, build a script that will make the requisite changes to the target database, execute that script.

Ryan was asking which of these one should use. I suggested that publishing was a better option and here are two reasons why:

1. Publish will include your pre and post deployment scripts as well whereas Schema Compare will not. (And if your retort is that you cannot run those scripts more than once then you’re doing it wrong, rewrite them.)
2. If the debug target for your project is configured correctly then a publish operation can be triggered simply by pressing F5. That’s massively more productive than the point-and-click nature of Schema Compare. Its even better if you have multiple SSDT projects in your solution as you can publish all of them with a single key stroke.

Does anyone out there have a different opinion? Let me know in the comments.

@Jamiet

In this week’s earlier blog post First release of my own personal T-SQL code library on Github I talked of how one could use a dacpac to distribute a bunch of code to different servers. Upon reading the blog post Jonathan Allen (of SQL Saturday Exeter fame), with whom I’ve been discussing dacpacs with on-and-off recently, sent me this email:

Hi Jamie,

The DacPac thing I emailed about in December hasnt taken off yet but I have just downloaded your code library to take a look and I like the way the dacpac works. Should I be able to open that in VS or is the dacpac compiled/built in VS? The video you linked to didnt cover dapac at all so I am in the dark on how to create one/them.

If I can build a database and create a dacpac simply then this could be really useful.

Jonathan’s email made me realise that there is perhaps a lot of confusion about what dacpacs are, what they can be used for and how they can be used so I figured a braindump of what I know about them might be useful, that’s what you’re getting in this blog post.

What is a dacpac?

A dacpac is a file with a .dacpac extension.

In that single file are a collection of definitions of objects that one could find in a SQL Server database such as tables, stored procedures, views plus some instance level objects such as logins too (the complete list of supported objects for SQL Server 2012 can be found at DAC Support For SQL Server Objects and Versions). The fact that a dacpac is a file means you can do anything you could do with any other file, store it, email it, share it on a file server etc… and this means that they are a great way of distributing the definition of many objects (perhaps even an entire database) in a single file. Or, as Microsoft puts it, a self-contained unit of SQL Server database deployment that enables data-tier developers and database administrators to package SQL Server objects into a portable artifact called a DAC package, also known as a DACPAC. That in itself is, I think, very powerful.

Ostensibly a dacpac is a binary file so you can’t just open it up in your favourite text editor and look at the contents of it. However, what many people do not know is that the format of a dacpac is simply the common ZIP compression format and hence we can add .zip to the end of a dacpac filename:

and open it up like you would any other .zip file to have a look inside. If you do so you will see this:

The contents of that zip file conform to something called the Open Packaging Convention (OPC). OPC is a standard defined by Microsoft for, well, for zipping up files basically. You have likely used files conforming to OPC before without knowing it, docx, .xlsx, .pptx are the most common ones that you might recognise if you use Microsoft Office and there are some more obscure ones such as .ispac (SSIS2012 developers should recognise that). (For a more complete list of OPC-compliant file types see the wikipedia page).

Notice in the screenshot above showing the innards of TSQLCodeLibrary.dacpac the biggest file is model.xml. This is the file that contains the definition of all our SQL Server objects. I won’t screenshot that here but I encourage you to get hold of a .dacpac file (here’s one) and have a poke around to see what’s in that model.xml file.

What are dacpacs for?

Dacpacs are used for deploying SQL Server objects to an instance of SQL Server. That’s it. If your job does not ever involve doing that then you probably don’t need to read any further.

Dacpac pre-requisites

A .docx file (i.e. A Microsoft Word document) isn’t much use to someone if they don’t have the software (i.e. Microsoft Word) to make use of it and so the analogy holds for dacpacs; in order to use them you need to have some software installed and that software is called the Data-tier Application Framework (or DAC Framework for short, or DacFx for even shorter).

Incidentally, you may be wondering what DAC stands for at this point. I think its “Data-Tier Application” in which case you may be thinking that the acronym DAC is a stupid one especially as DAC also stands for something else in SQL Server, I would agree!

DacFx is available to download for free however you’ll probably never need to do that as installation of DacFX occurs whenever you install SQL Server, SQL Server client tools or SQL Server Data Tools (SSDT). If DacFX is installed you should be able to see it in Programs and Features:

How does one deploy a dacpac?

In dacpac nomenclature the correct term for deploying a dacpac is publishing however the two generally get used interchangeably. There are two methods of publishing a dacpac which I’ll cover below.

Publish via SSMS

In SSMS’s Object Explorer right-click on the databases node and select “Deploy Data-tier Application…” (told you they used those terms interchangeably):

This launches a wizard that prompts you to choose a dacpac, fill in some particulars (e.g. database name) and then deploy it for you by calling out to DacFx. Unfortunately this wizard is not very good because it doesn’t (currently) support all features of dacpacs, namely if your dacpacs contain any sqlcmd variables (I won’t cover those here but they are commonly used within dacpacs) a value needs to be supplied for them; the wizard doesn’t prompt you for a value and hence the deployment fails.

This. Is. Stupid. Microsoft should be suitably lambasted for not providing this basic functionality. Anyway, due to this limitation you’re most likely to be using the other method which is…

Publish via command-line

One component distributed in DacFx is a command-line tool called sqlpackage.exe which will quickly become your best friend if you use dacpacs a lot. sqlpackage.exe can do a lot of things and those “things” are referred to as actions, one of those actions is publishing a dacpac. Here’s the syntax for publishing a dacpac using sqlpackage.exe:

"%ProgramFiles(x86)%\Microsoft SQL Server\110 \DAC\bin\SqlPackage.exe"
      /action:Publish
      /SourceFile:<path to your dacpac>
      /TargetServerName:<SQL instance you are deploying to>
      /TargetDatabaseName:<Name of either (a)the database to create or (b) the existing database to deploy into>

Publishing is idempotent

Notice from my comment above for TargetDatabaseName that you can deploy to an existing database. You might ask why you might want to publish into an existing database, after all, the objects you are publishing might already exist. This segues nicely into what I see as the biggest benefit of dacpacs and DacFx, the software interrogates the target database to determine whether or not the objects already exist or not and if they do not it will create them. If they do already exist it will determine whether the definition has changed or not and if it has, it will make those changes. DacFx will always protect your data so if it determines that an operation would cause data destruction (e.g. removing a column from a table) then it will (optionally) throw an error and fail. You never again need to write an ALTER statement or first check that an object exists in order to change an object definition, DacFx will do it for you. To put it another way, publishing using dacpacs and DacFx is idempotent.

How does one create a dacpac?

Of course in order to publish a dacpac you’re first going to have to create one and one of Jonathan’s questions above pertained to exactly this. There are two distinct ways to do create a dacpac.

Use an SSDT Project

SQL Server Data Tools (SSDT) projects are basically a project type within Visual Studio that provide a way of building DDL for SQL Server databases. I’m not going to cover SSDT projects in any detail here except to say that when such a project is built the output is a dacpac. Note that SSDT can also publish the dacpac for you however I didn’t mention that above as the publish operation is essentially another wrapper around the same DacFx functionality used by sqlpackage.exe.

Create from an existing database

One can right-click on a database in SSMS and click on “Extract Data-tier Application…” to create a dacpac containing the definition of all objects in that database:

Wrap-up

Should you be using dacpacs? I can’t answer that question for you but hopefully what I’ve done is given you enough information so that you can answer it for yourself. Some people might like the way dacpacs encapsulate many objects into a single file and their idempotent deployment, others may prefer good old simple, handcrafted T-SQL scripts which don’t have any pre-requisites other than SQL Server itself. The choice is yours.

Further reading

• Data-tier Applications on Technet

Release-to-manufacturing of SQL Server 2014 was announced today and with it comes some welcome news about some small enhancements to SSDT:

The SSDT team has been heads down getting our SQL Server 2014 release ready to go.  However we've made time to make some targeted improvements in this area.  In our upcoming release for SQL Server 2014 the Change Connection part of this blog will be implemented.  The F6 functionality (included moving between multiple result sets) didn't make it, but will be in the update after that. – Patrick Sirr

http://sqlblog.com/blogs/jamie_thomson/archive/2013/03/19/connected-development-in-ssdt-versus-ssms.aspx#53271

and

I've implemented F6 and shift-F6 to cycle through the tabs as SSMS does (including for multiple result sets). However the fix won't make it into our upcoming SQL Server 2014 release due in a couple weeks. It'll be in the next update.

http://connect.microsoft.com/SQLServer/feedback/details/780990/ssdt-f6-to-move-between-panes-in-a-query-window

Hopefully there’ll be a few more enhancements in the SSDT Connected Development arena other than the two mentioned here (more scripting options would be nice).

Who says submitting to Microsoft Connect never works?

Don’t forget that you don’t need to have SQL Server 2014 in order to get these new features. SSDT is free and supports all versions back to SQL Server 2005.

@Jamiet

An issue that regularly seems to rear its head on my travels is that of headless build servers for SSDT. What does that mean exactly? Let me give you my interpretation of it.

A SQL Server Data Tools (SSDT) project incorporates a build process that will basically parse all of the files within the project and spit out a .dacpac file. Where an organisation employs a Continuous Integration process they will likely want to automate the building of that dacpac whenever someone commits a change to the source control repository. In order to do that the organisation will use a build server (e.g. TFS, TeamCity, Jenkins) and hence that build server requires all the pre-requisite software that understands how to build an SSDT project.

The simplest way to install all of those pre-requisites is to install SSDT itself however a lot of folks don’t like that approach because it installs a lot unnecessary components on there, not least Visual Studio itself. Those folks (of which i am one) are of the opinion that it should be unnecessary to install a heavyweight GUI in order to simply get a few software components required to do something that inherently doesn’t even need a GUI. The phrase “headless build server” is often used to describe a build server that doesn’t contain any heavyweight GUI tools such as Visual Studio and is a desirable state for a build server.

In his blog post Headless MSBuild Support for SSDT (*.sqlproj) Projects Gert Drapers outlines the steps necessary to obtain a headless build server for SSDT:

This article describes how to install the required components to build and publish SQL Server Data Tools projects (*.sqlproj) using MSBuild without installing the full SQL Server Data Tool hosted inside the Visual Studio IDE.
http://sqlproj.com/index.php/2012/03/headless-msbuild-support-for-ssdt-sqlproj-projects/

Frankly however going through these steps is a royal PITA and folks like myself have longed for Microsoft to support headless build support for SSDT by providing a distributable installer that installs only the pre-requisites for building SSDT projects. Yesterday in MSDN forum thread Building a VS2013 headless build server - it's sooo hard Mike Hingley complained about this very thing and it prompted a response from Kevin Cunnane from the SSDT product team:

The official recommendation from the TFS / Visual Studio team is to install the version of Visual Studio you use on the build machine.

I, like many others, would rather not have to install full blown Visual Studio and so I asked:

Is there any chance you'll ever support any of these scenarios:

• Installation of all build/deploy pre-requisites without installing the VS shell?
• TFS shipping with all of the pre-requisites for doing SSDT project build/deploys
• 3rd party build servers (e.g. TeamCity) shipping with all of the requisites for doing SSDT project build/deploys

I have to say that the lack of a single installer containing all the pre-requisites for SSDT build/deploy puzzles me. Surely the DacFX installer would be a perfect vehicle for that?

Kevin replied again:

The answer is no for all 3 scenarios. We looked into this issue, discussed it with the Visual Studio / TFS team, and in the end agreed to go with their latest guidance which is to install Visual Studio (e.g. VS2013 Express for Web) on the build machine. This is how Visual Studio Online is doing it and it's the approach recommended for customers setting up their own TFS build servers. I would hope this is compatible with 3rd party build servers but have not verified whether this works with TeamCity etc.

Note that DacFx MSI isn't a suitable release vehicle for this as we don't want to include Visual Studio/MSBuild dependencies in that package. It's meant to just include the core DacFx DLLs used by SSMS, SqlPackage.exe on the command line, etc.

What this means is we won't be providing a separate MSI installer or nuget package with just the necessary build DLLs you need to run your build and tests. If someone wanted to create a script that generated a nuget package based on our DLLs and targets files, then release that somewhere on the web for easier integration with 3rd party build servers we've no problem with that.

Again, here’s the link to the thread and its worth reading in its entirety if this is something that interests you.

So there you have it. Microsoft will not be be providing support for headless build servers for SSDT but if someone in the community wants to go ahead and roll their own, go right ahead.

@Jamiet

Like many folks I’m a keen podcast listener and yesterday my commute was filled by listening to Scott Hunter being interviewed on .Net Rocks about the next version of ASP.Net. One thing Scott said really struck a chord with me. I don’t remember the full quote but he was talking about how the ASP.Net project file (i.e. the .csproj file) is going away. The rationale being that the main purpose of that file is to list all the other files in the project, and that’s something that the file system is pretty good at. In Scott’s own words (that someone helpfully put in the comments):

A file that lists files is really redundant when the OS already does this

Romeliz Valenciano correctly pointed out on Twitter that there will still be a project.json file however no longer will there be a need to keep a list of files in a project file. I suspect project.json will simply contain a list of exclusions where necessary rather than the current approach where the project file is a list of inclusions.

On the face of it this seems like a pretty good idea. I’ve long been a fan of convention over configuration and this is a great example of that. Instead of listing all the files in a separate file, just treat all the files in the directory as being part of the project. Ostensibly the approach is if its in the directory, its part of the project. Simple.

Now I’m not an ASP.net developer, far from it, but it did occur to me that the same approach could be applied to the two Visual Studio project types that I am most familiar with, SSIS & SSDT. Like many people I’ve long been irritated by SSIS projects that display a faux file system inside Solution Explorer. As you can see in the screenshot below the project has Miscellaneous and Connection Managers folders but no such folders exist on the file system:

This may seem like a minor thing but it means useful Solution Explorer features like Show All Files and Open Folder in Windows Explorer don’t work and quite frankly it makes me feel like a second class citizen in the Microsoft ecosystem. I’m a developer, treat me like one. Don’t try and hide the detail of how a project works under the covers, show it to me. I’m a big boy, I can handle it!

Would it not be preferable to simply treat all the .dtsx files in a directory as being part of a project? I think it would, that’s pretty much all the .dtproj file does anyway (that, and present things in a non-alphabetic order – something else that wildly irritates me), so why not just get rid of the .dtproj file?

In the case of SSDT the .sqlproj actually does a whole lot more than simply list files because it also states the BuildAction of each file (Build, NotInBuild, Post-Deployment, etc…) but I see no reason why the convention over configuration approach can’t help us there either. Want to know which is the Post-deployment script? Well, its the one called Post-DeploymentScript.sql! Simple!

So that’s my new crusade. Let’s kill all the project files (well, the .dtproj & .sqlproj ones anyway). Are you with me?

@Jamiet