Tuesday, August 10, 2010

WCF Architecture

The following figure illustrates the major components of WCF.

Figure 1: WCF Architecture

Contracts

Contracts layer are next to that of Application layer. Developer will directly use this contract to develop the service. We are also going to do the same now. Let us see briefly what these contracts will do for us and we will also know that WCF is working on message system.

Service contracts

- Describe about the operation that service can provide. Example, Service provided to know the temperature of the city based on the zip code, this service we call as Service contract. It will be created using Service and Operational Contract attribute.

Data contract

- It describes the custom data type which is exposed to the client. This defines the data types, are passed to and from service. Data types like int, string are identified by the client because it is already mention in XML schema definition language document, but custom created class or datatype cannot be identified by the client e.g. Employee data type. By using DataContract we can make client aware that we are using Employee data type for returning or passing parameter to the method.

Message Contract

- Default SOAP message format is provided by the WCF runtime for communication between Client and service. If it is not meeting your requirements then we can create our own message format. This can be achieved by using Message Contract attribute.

Policies and Binding

- Specify conditions required to communicate with a service e.g security requirement to communicate with service, protocol and encoding used for binding.

Service Runtime

- It contains the behaviors that occur during runtime of service.

  • Throttling Behavior- Controls how many messages are processed.
  • Error Behavior - Specifies what occurs, when internal error occurs on the service.
  • Metadata Behavior - Tells how and whether metadata is available to outside world.
  • Instance Behavior - Specifies how many instance of the service has to be created while running.
  • Transaction Behavior - Enables the rollback of transacted operations if a failure occurs.
  • Dispatch Behavior - Controls how a message is processed by the WCF Infrastructure.

Messaging

- Messaging layer is composed of channels. A channel is a component that processes a message in some way, for example, by authenticating a message. A set of channels is also known as a channel stack. Channels are the core abstraction for sending message to and receiving message from an Endpoint. Broadly we can categories channels as

  • Transport Channels

    Handles sending and receiving message from network. Protocols like HTTP, TCP, name pipes and MSMQ.

  • Protocol Channels

    Implements SOAP based protocol by processing and possibly modifying message. E.g. WS-Security and WS-Reliability.

Activation and Hosting

- Services can be hosted or executed, so that it will be available to everyone accessing from the client. WCF service can be hosted by following mechanism

  • IIS

    Internet information Service provides number of advantages if a Service uses Http as protocol. It does not require Host code to activate the service, it automatically activates service code.

  • Windows Activation Service

    (WAS) is the new process activation mechanism that ships with IIS 7.0. In addition to HTTP based communication, WCF can also use WAS to provide message-based activation over other protocols, such as TCP and named pipes.

  • Self-Hosting

    WCF service can be self hosted as console application, Win Forms or WPF application with graphical UI.

  • Windows Service

    WCF can also be hosted as a Windows Service, so that it is under control of the Service Control Manager (SCM).

Tuesday, April 27, 2010

Processing Data Queues in SQL Server with READPAST and UPDLOCK

Problem
One common processing problem that a DBA can encounter is processing rows from a table used as a data queue. Queue processing is functionality where one or more processes INSERTs rows into a database table representing a work queue with each row representing a business action that must be performed. At the same time, one or more processes SELECTs records from the same queue table in order to execute the business action required by the application while later deleting the processed row so it is not processed again. Typically, the reading processes use polling to interrogate the queuing table for any new rows that require execution of a business action. If done incorrectly, processing data queues can produce unexpected results and/or performance issues.

Solution
The following examples set up a process queue to be processed by two separate processes reading the same queue. This can be extended beyond two processes, but for this example we want to show you how two processes can work against one work queue. We’ll start with examples to illustrate issues that can be encountered.

First, let’s create a sample table and populate it with 10 records to be processed.

-- create an example queue table
CREATE TABLE DBO.QUEUE (
QUEUEID INT IDENTITY( 1 , 1 ) NOT NULL PRIMARY KEY,
SOMEACTION VARCHAR(100))

GO

-- seed the queue table with 10 rows
DECLARE @counter INT

SELECT
@counter = 1

WHILE (@counter <= 10)
BEGIN
INSERT INTO
DBO.QUEUE
(SOMEACTION)
SELECT 'some action ' + CAST(@counter AS VARCHAR)

SELECT @counter = @counter + 1
END

Encountering unexpected results
Open 2 separate query windows and issue the following statements in each session:

DECLARE @queueid INT

BEGIN TRAN
TRAN1

SELECT TOP 1 @queueid = QUEUEID
FROM DBO.QUEUE

PRINT 'processing queueid # ' + CAST(@queueid AS VARCHAR)

-- account for delay in processing time
WAITFOR DELAY '00:00:10'

DELETE FROM DBO.QUEUE
WHERE QUEUEID = @queueid

COMMIT

As you will see, each session processed the same row! This is obviously unacceptable processing behavior, but what can we do about it?

We can eliminate this behavior by adding the UPDLOCK hint to the SELECT statement. The UPDLOCK hint tells the SQL Server query engine “Don’t allow any other reader of this row to acquire an UPDLOCK (“U” lock) because I will be promoting this lock to an exclusive “X” lock later in my processing”. It effectively reserves the row for your processing. However, as you will see, this can cause a new problem to arise.

Encountering blocking
The SELECT statement has been modified to use the UPDLOCK hint.

Open 2 separate query windows and issue the following statements again.

DECLARE @queueid INT

BEGIN TRAN
TRAN1

SELECT TOP 1 @queueid = QUEUEID
FROM DBO.QUEUE WITH (updlock)

PRINT 'processing queueid # ' + CAST(@queueid AS VARCHAR)

-- account for delay in processing time
WAITFOR DELAY '00:00:10'

DELETE FROM DBO.QUEUE
WHERE QUEUEID = @queueid

COMMIT

As you can see from the modified example, each session now processes separate rows. Good so far. However, the 2nd session took longer to execute than it did in the first example even though it now processes a separate row. Why is this? It’s because an UPDLOCK (“U”) lock has been placed on the row processed by the first session and the 2nd session is forced to wait on this lock to be released before it is allowed to retrieve the next row for processing. This is highly inefficient since multiple consumers of the queue must all wait until any locks are released. So, how do we get around this?

To get around the blocking encountered in the previous example, a READPAST hint can be used in conjunction with the UPDLOCK hint. The READPAST hint tells the SQL Server query engine “If you encounter any rows that are locked, just skip them… I want whatever is not currently being processed by anyone”.

Incorporating the READPAST query hint
The SELECT statement has been modified to use the READPAST hint in addition to the UPDLOCK hint.

Open 2 separate query windows and issue the following statements again.

DECLARE @queueid INT

BEGIN TRAN
TRAN1

SELECT TOP 1 @queueid = QUEUEID
FROM DBO.QUEUE WITH (updlock, readpast)

PRINT 'processing queueid # ' + CAST(@queueid AS VARCHAR)

-- account for delay in processing time
WAITFOR DELAY '00:00:10'

DELETE FROM DBO.QUEUE
WHERE QUEUEID = @queueid

COMMIT


As you can see from this latest example, each session now processes separate rows and the 2nd session is no longer blocked as evidenced by the execution time (both sessions should complete at roughly the same time).

Using the UPDLOCK hint in conjunction with the READPAST hint gives the best performance for processing queues while eliminating unexpected results and blocking.

Putting it all together
Here is an example of the above code that takes it a step further and processes all of the records in the queue. To run this drop table dbo.queue and then recreate it by running the code in the very first code block above that creates the table and loads the data.

Open 2 separate query windows and issue the following statements again.

SET NOCOUNT ON
DECLARE
@queueid INT

WHILE
(SELECT COUNT(*) FROM DBO.QUEUE WITH (updlock, readpast)) >= 1

BEGIN

BEGIN TRAN
TRAN1

SELECT TOP 1 @queueid = QUEUEID
FROM DBO.QUEUE WITH (updlock, readpast)

PRINT 'processing queueid # ' + CAST(@queueid AS VARCHAR)

-- account for delay in processing time
WAITFOR DELAY '00:00:05'

DELETE FROM DBO.QUEUE
WHERE QUEUEID = @queueid
COMMIT
END

Next Steps

  • When processing data queues, use the UPDLOCK hint along with the READPAST hint to get maximum throughput of your data queues.
  • Read more information about UPDLOCK and READPAST in the SQL Server 2000 and 2005 Books Online under Locking Hints.
  • Read more about Lock Compatibility in the SQL Server 2000 and 2005 Books Online
  • Thank you to Armando Prato for providing this tip!

Wednesday, March 24, 2010

VS2010 Tutorial: Testing Tutorial (Step 2)

Last time I posted the Step 1 tutorial for creating a WPF application from scratch using Visual Studio 2010. In this post, I’m going to create a test plan and test case for the application using the new Visual Studio Test and Lab Manager project.

For this tutorial, I’m using Visual Studio Team Suite 2010 (which includes all of the roles and TFS access). I’ve already added the demo to TFS so I have full source control. For the sake of demonstration, I’ve commented out the final fix from the walk through so the label does not update:

image

When I run the application, the label is not updated:

image

The Tester

To create my test, I’m going to run the Test and Lab Manager tool from the start menu:

image

The main page has tabs for test plans, tests, and for tracking work items:

image

First I need to connect to my TFS server by click Add. My server is VLM13267036 (auto generated name by our internal Hyper-V testing tools):

image

I’ve already got a collection with my code stored in the Projects folder:

image

Next I’ll select Projects and choose the Connect option. This prompts me to set a context:

image

I’ll choose “Set context” which brings up the editor for my new context:

image

I’ll select New:

image

I can now edit all of the properties:

image

After all data has been entered, click Save and Close. The new item is now in our list:

image

Double clicking the item allows me to add a new test case to this test suite:

image

Here you can fill out all details for the test case:

image

Steps can now be added by clicking on the “Click here to add a step”:

image

I’ve added a few steps including launching the application, hitting the buttons, etc:

image

Now hit Save and Close to go back to the test case list. The Plan is now complete. We can run it any time a new build is produced, for each flavor of build, for different configurations, etc. To execute the test, change focus to the Test tab:

image

Our test plan and test case are already in the list. Right click the test case and select Run:

image

This will launch the test case. The manual test runner window docks itself to the left side of my desktop so I can see both the steps I want to run and the full Windows desktop:

image

The “Start test and record” option allows me to not only do the testing steps, but it will also allow recording a WMV of all the steps I do as well as recording my steps to help me author coded UI tests (big helper with automation). This is really handy if you want someone to see exactly what you did to produce a bug and automate testing in the future.

In this case I will select “Start test”. Notice the Test Runner now shows the steps I created above:

image

The first step is to “Launch the PicViewer application” which I’ll do by running the application. Since that worked, I’ll press the combo box status item behind the step and select ‘Pass’ from the drop down:

image

The item is now marked as passing:

image

I’ll repeat the process for the next two steps, so far so good:

image

When I get to my last step, I discover the file path isn’t actually set. That makes this item a failure. Select the drop down box and choose ‘Fail’ from the list. I’m automatically asked for comments on the failure:

image

Since I didn’t record a video of my steps, I would like to give the developer a screen shot of what went wrong. Select the camera tool bar button:

image

This will bring up a capture tool turning the cursor into a cross hair that allows me to select a region of the screen. I’ll select the top of the application to demonstrate the busted label:

image

Notice that the failed test now has a .png file added with the image:

image

I’ve got enough supporting data now so I’ll create a new bug using the toolbar:

image

I’m now prompted to create my bug with a description, etc:

image

Notice the detailed test steps I’ve taken have already been added to the bug:

image

As has my screen shot (the .png file):

image

I’ll now do a Save and Close which will commit the bug to TFS as a Work Item. Finally I’ll do End Test then Save and Close the test runner. This will return us to Test and Lab Manager.

image

as a tester, I could now double click the test case and see all of the same data I just saved for the failure:

image

I can also select My Bugs and see the bug filed for this issue (since I conveniently assigned it to me):

image

And just to show how everything is wired together, I can open Visual Studio, Team Explorer and look for bugs assigned to me there as well:

image

At this point my job as a tester is now done.

The Developer

As a developer, I can now open the bug and read through the issue. If I select Other Links I’ll find the .png which I can open to see the issue:

image

Sure enough, the label is not updated. If a WMV had been recorded, I could have actually watched the testing steps in action. Because the bug is quite simple to find and fix (some idiot commented out the update line!) I can simply make my fix and test it.

Now that things are fixed, I want to check in the bug fix and resolve the work item at the same time. Click on the Pending Changes tab in VS and select the correct work item:

image

Now we can Check In the fix:

image

I can now verify the bug has been Resolved:

image

In Summary

A key goal for Visual Studio Team System 2010 was to reduce the number of times a tester and developer wind up in a ‘no-repro’ situation. There are several things I’ve demonstrated in this tutorial which help:

  • Test case steps are documented and set up for repeatable execution
  • Pass/Fail steps are outlined and stored in bugs automatically
  • Video capture is allowed to see all steps taken, and screen snapshots are easy to acquire and file with a bug
  • System information including build number, OS, etc are recorded for you (System Info tab)
  • Although not shown, I could also have collected all of the historical debugging traces from the run as well
  • All data from test cases, results, work items, and source code are kept in TFS and can be shared by test and dev

I hope you’ll pick up Beta 1 and try this set of tutorials for yourself. Let us know how well it works for you and if you have any suggestions. I should also point out the work item tracking, auto resolve, etc are all part of VS 2008 so a great way to get prepared for the new version is to get TFS deployed today and get your projects into the system.

Enjoy!

VS2010 Tutorial: Build a WPF App (Step 1)

For this post I’m going to build a WPF application from scratch using VS2010 Beta 1. Along the way I’m going to show off a few new features of the product. I built this demo using the Beta 1 build on Windows 7 RC, but it should work fine on other systems as well.

If you haven’t already downloaded the software, you can find a great walk through of how to do that on Channel 9 from Brian Keller. Brian explains how to install the entire product, including TFS. TFS is not required for Step 1 (this tutorial), but will be for Step 2.

First we’ll start off in Visual Studio 2010 and do a File, New Project. I’m going to select “WPF Application” and call my application PicViewer (this is the new Beta 1 dialog):

image

This will give us the default project template including a Window with a Grid and my default C# project code:

image

To start, I’m going to drag the Window out to be larger which you can do by dragging the actual window in the designer or you can edit the XAML Width=”” tag. I’ll make mine 700:

image

At the top of the application we want a button to display the Previous and Next pictures in the list. We also want to display the full path so we’ll need a label. Step 1 is to add a row to the grid. In this case I’ll select the label and move the cursor to where I want the row:

image

When I make the row, it also adjusts the XAML for this page as follows:

image

Let’s adjust the first row Height to “35” instead of “29*” just to make sure my buttons are of reasonable size. The aesthetics are really up to you.

We want WPF to do the layout of the buttons and the label. To do this, create a bound to Grid.Row=”0”. The StackPanel by default flows items vertically so let’s change the Orientation to “Horizontal”. You can do this either by using the XAML editor or by dragging the items from the ToolBox. Your final markup should look like the following:

image

Let’s drag a button onto the Row from the Toolbox:

image

I want this to be the Previous button. I’ll do this by editing the settings in the Properties window:

image

First change the Name field to “buttonPrev” and then change the Content setting to “Prev”. Repeat this process by dragging another button to the right of the Prev button and edit that item to Name “buttonNext” and Content “Next”. You should see the following:

image

This is functional, but doesn’t look good. We can add some white space around the buttons in the StackPanel by setting the margin which defines how much space should be around an item. Select the Prev button and change this setting:

image

which will give the button a Margin of 5 on all sides (you can independently set left|right|top|bottom as well). Repeat the same for the Next button. At this point you’ll see the following:

image

Finally drag a label to the right of the Next button making the Name “labelPath” and set the Width to “400”. When you are done, your XAML markup should look as follows (you can edit the text if it isn’t just right):

image

To display our pictures, we will need to have an Image control in the bottom. To do this, drag an Image control from the ToolBox to the bottom of the grid. This will do the following:

image

This is a case where having the Smart Tag feature of Windows Forms would be really handy. This is on the feature list but not yet in the WPF Designer. In this case I’m going to edit the XAML itself to remove everything the designer added except the Grid.Row, Margin, and Name settings. Make your settings look like the following and the image control will now fill the bottom of the grid with a nice white space border:

image

image

Now the look of the application is coming along and we need to add some code to make it work. To start, double click on the title bar of the Window to create a load handler:

image

image

Repeat this process by double clicking on the Prev and Next buttons. This will give you the following code outline:

image

For this picture viewer, I’m going to get a list of jpg files from my Windows 7 user directory and display those. What I’d really like to do is write the view code in Window1.xaml.cs in a way that makes sense to me, then go write the underlying logic. To get started, I’m going to declare a new instance of a class that doesn’t exist yet:

image

Because the class doesn’t exist yet, I am getting the red squiggles under the type. That’s ok, we’ll create the class in a minute.

The next step is to stub out all the view code in this file. We’ll start with the Window_Loaded() method. Notice that even though the class has not been defined, Intellisense will include the type after the new keyword so you can use it:

image

We want to initialize the list and display the first picture when the application starts. The code is as follows:

image

Once again I have parse errors for items which don’t exist which is ok. Move on to the Previous button handler and start typing in that method. When I get to the DisplayPicture function call, you’ll notice the function doesn’t show up in the IntelliSense window because it doesn’t exist yet.

image

If you hit ++Space you’ll change the mode into “consume first” which means I want the editor to give me these names while I’m typing. Now when I start typing, my new (and undeclared) function shows up in the list so IntelliSense doesn’t auto-complete on items already in the completion list:

image

My code then looks like the following:

image

I’m happy with this basic view logic. Now I need to actually create the support class that finds my pictures. Let’s start by placing the cursor on the PictureList class declaration then press the key and the period (+.) at the same time to expand the smart tag. You’ll get the following pop up:

image

Select the first menu item, “Generate class for ‘PictureList’”. This will add a new file, PictureList.cs, to the project and generate the class PictureList in that file. Now that the class has been defined, the squiggles go away:

image

You will then notice all the methods you have invoked on the class are showing up as syntax errors. Put your cursor in the Init() method and press +. to get the following:

image

Select the menu item so the code is generated for you. Repeat this process for the Peek(), Prev(), and Next() methods. If we then open PictureList.cs file, you will find the following code stubbed out for you:

image

Notice that because the code we wrote to consume methods like _PicList.Peek() expect a string back, the editor has automatically given the methods a return type of ‘string’.

Now I’ll add the logic to retrieve a list of jpg’s from the Windows install and give me the ability to walk through them:

image

The last step is to write the DisplayPicture method in Window1.xaml.cs. I can do this once again by selecting the method name and hitting +. and then fill out the code as follows:

image

This code will take the jpg and convert it into a BitmapImage that can be displayed. The next line of code sets the Source of our XAML image contorl to this bitmap so it can be displayed on the screen.

The program now builds cleanly. I will set a breakpoint in the Peek() method as well as the DisplayPicture method. Do this by placing the cursor on those functions and hitting the F9 button (notice the red breakpoint circle in the margin):

image

image

Now let’s run the application by hitting the F5 button. Our first breakpoint in the Peek() method is now hit:

image

You’ll find your breakpoint marked in yellow where we stopped. In addition you’ll find the traditional Call Stack view in the lower right. Because I’m using Visual Studio Team System, I also have the “Debug History” view on by default:

image

The default settings have captured interesting information about the execution of my program so far including all of the Registry accesses my program has made (either by me or the underlying system) and system events (like breakpoints, exceptions, etc).

If I hover my cursor over the _PicList variable in the editor, I get a pop up watch window. If I hover my cursor over the little square at the end and click:

image

then you will get a sticky DataTips window in the editor (the arrow is just to demonstrate the result, it isn’t in the editor):

image

This is a simple example of using the new WPF editor. The sticky DataTips window is a WPF adornment to the text buffer tied into the debugger. You can use it like a normal watch window by expanding the array contents, etc.

image

Let’s hit F5 and continue the program. My application now works including the Prev and Next buttons:

image

The only issue is the name of the file is not being updated. Click the Next button again to our break point in DisplayPicture(). There is no code to update the label in this method:

image

To fix this, let’s add a new line of code:

image

With the Edit & Continue feature you can make new code changes while the application is running. Update the code to the following:

image

Now press F5 to continue and the path is updated:

image

There are a couple of fit and finish things I’d like to change. Let’s stretch the image to fit by adding Stretch=”Fill”:

image

Next let’s add a rectangle to frame the picture:

image

You can then select the rectangle and set the Fill property to give it a color. This brings up the new inline color palate:

image

I chose a blue/black color and you get the following:

image

The final XAML markup then looks like the following:

image

This sample could be improved in a number of ways, including using data bound controls to enumerator classes, adding all the proper try/catch logic, etc. But my main goal here was to show off some editor / designer features so I’ll leave that as an exercise to the user.

Enjoy!